We propose a novel and challenging benchmark, AutoEval-Video, to comprehensively evaluate large vision-language models in open-ended video question answering. The comprehensiveness of AutoEval-Video is demonstrated in two aspects: 1) AutoEval-Video constructs open-ended video-questions across 9 skill dimensions, addressing capabilities of perception, comprehension, and generation. 2) AutoEval-Video contains newly collected videos that cover over 40 distinct themes. To efficiently evaluate responses to the open-ended questions, we employ an LLM-based evaluation approach, but instead of merely providing a reference answer, we annotate unique evaluation rules for every single instance (video-question pair). To maximize the robustness of these rules, we develop a novel adversarial annotation mechanism. By using instance-specific rules as prompt, GPT-4, as an automatic evaluator, can achieve a stable evaluation accuracy of around 97.0%, comparable to the 94.9% - 97.5% accuracy of a human evaluator. Furthermore, we assess the performance of eight large vision-language models on AutoEval-Video. Among them, GPT-4V(ision) significantly outperforms other models, achieving an accuracy of 32.2%. However, there is still substantial room for improvement compared to human accuracy of 72.8%. By conducting an extensive case study, we uncover several drawbacks of GPT-4V, such as limited temporal and dynamic comprehension, and overly general responses. Code is available at https://github.com/Xiuyuan-Chen/AutoEval-Video.
Person retrieval has attracted rising attention. Existing methods are mainly divided into two retrieval modes, namely image-only and text-only. However, they are unable to make full use of the available information and are difficult to meet diverse application requirements. To address the above limitations, we propose a new Composed Person Retrieval (CPR) task, which combines visual and textual queries to identify individuals of interest from large-scale person image databases. Nevertheless, the foremost difficulty of the CPR task is the lack of available annotated datasets. Therefore, we first introduce a scalable automatic data synthesis pipeline, which decomposes complex multimodal data generation into the creation of textual quadruples followed by identity-consistent image synthesis using fine-tuned generative models. Meanwhile, a multimodal filtering method is designed to ensure the resulting SynCPR dataset retains 1.15 million high-quality and fully synthetic triplets. Additionally, to improve the representation of composed person queries, we propose a novel Fine-grained Adaptive Feature Alignment (FAFA) framework through fine-grained dynamic alignment and masked feature reasoning. Moreover, for objective evaluation, we manually annotate the Image-Text Composed Person Retrieval (ITCPR) test set. The extensive experiments demonstrate the effectiveness of the SynCPR dataset and the superiority of the proposed FAFA framework when compared with the state-of-the-art methods. All code and data will be provided at https://github.com/Delong-liu-bupt/Composed_Person_Retrieval.
Recently, remote sensing image captioning has gained significant attention in the remote sensing community. Due to the significant differences in spatial resolution of remote sensing images, existing methods in this field have predominantly concentrated on the fine-grained extraction of remote sensing image features, but they cannot effectively handle the semantic consistency between visual features and textual features. To efficiently align the image-text, we propose a novel two-stage vision-language pre-training-based approach to bootstrap interactive image-text alignment for remote sensing image captioning, called BITA, which relies on the design of a lightweight interactive Fourier Transformer to better align remote sensing image-text features. The Fourier layer in the interactive Fourier Transformer is capable of extracting multi-scale features of remote sensing images in the frequency domain, thereby reducing the redundancy of remote sensing visual features. Specifically, the first stage involves preliminary alignment through image-text contrastive learning, which aligns the learned multi-scale remote sensing features from the interactive Fourier Transformer with textual features. In the second stage, the interactive Fourier Transformer connects the frozen image encoder with a large language model. Then, prefix causal language modeling is utilized to guide the text generation process using visual features. Ultimately, across the UCM-caption, RSICD, and NWPU-caption datasets, the experimental results clearly demonstrate that BITA outperforms other advanced comparative approaches. The code is available at https://github.com/yangcong356/BITA.
Realistic video simulation has shown significant potential across diverse applications, from virtual reality to film production. This is particularly true for scenarios where capturing videos in real-world settings is either impractical or expensive. Existing approaches in video simulation often fail to accurately model the lighting environment, represent the object geometry, or achieve high levels of photorealism. In this paper, we propose Anything in Any Scene, a novel and generic framework for realistic video simulation that seamlessly inserts any object into an existing dynamic video with a strong emphasis on physical realism. Our proposed general framework encompasses three key processes: 1) integrating a realistic object into a given scene video with proper placement to ensure geometric realism; 2) estimating the sky and environmental lighting distribution and simulating realistic shadows to enhance the light realism; 3) employing a style transfer network that refines the final video output to maximize photorealism. We experimentally demonstrate that Anything in Any Scene framework produces simulated videos of great geometric realism, lighting realism, and photorealism. By significantly mitigating the challenges associated with video data generation, our framework offers an efficient and cost-effective solution for acquiring high-quality videos. Furthermore, its applications extend well beyond video data augmentation, showing promising potential in virtual reality, video editing, and various other video-centric applications. Please check our project website https://anythinginanyscene.github.io for access to our project code and more high-resolution video results.
Semantic segmentation in bird's eye view (BEV) plays a crucial role in autonomous driving. Previous methods usually follow an end-to-end pipeline, directly predicting the BEV segmentation map from monocular RGB inputs. However, the challenge arises when the RGB inputs and BEV targets from distinct perspectives, making the direct point-to-point predicting hard to optimize. In this paper, we decompose the original BEV segmentation task into two stages, namely BEV map reconstruction and RGB-BEV feature alignment. In the first stage, we train a BEV autoencoder to reconstruct the BEV segmentation maps given corrupted noisy latent representation, which urges the decoder to learn fundamental knowledge of typical BEV patterns. The second stage involves mapping RGB input images into the BEV latent space of the first stage, directly optimizing the correlations between the two views at the feature level. Our approach simplifies the complexity of combining perception and generation into distinct steps, equipping the model to handle intricate and challenging scenes effectively. Besides, we propose to transform the BEV segmentation map from the Cartesian to the polar coordinate system to establish the column-wise correspondence between RGB images and BEV maps. Moreover, our method requires neither multi-scale features nor camera intrinsic parameters for depth estimation and saves computational overhead. Extensive experiments on nuScenes and Argoverse show the effectiveness and efficiency of our method. Code is available at https://github.com/happytianhao/TaDe.
As blockchain technology rapidly evolves, the demand for enhanced efficiency, security, and scalability grows.Transformer models, as powerful deep learning architectures,have shown unprecedented potential in addressing various blockchain challenges. However, a systematic review of Transformer applications in blockchain is lacking. This paper aims to fill this research gap by surveying over 200 relevant papers, comprehensively reviewing practical cases and research progress of Transformers in blockchain applications. Our survey covers key areas including anomaly detection, smart contract security analysis, cryptocurrency prediction and trend analysis, and code summary generation. To clearly articulate the advancements of Transformers across various blockchain domains, we adopt a domain-oriented classification system, organizing and introducing representative methods based on major challenges in current blockchain research. For each research domain,we first introduce its background and objectives, then review previous representative methods and analyze their limitations,and finally introduce the advancements brought by Transformer models. Furthermore, we explore the challenges of utilizing Transformer, such as data privacy, model complexity, and real-time processing requirements. Finally, this article proposes future research directions, emphasizing the importance of exploring the Transformer architecture in depth to adapt it to specific blockchain applications, and discusses its potential role in promoting the development of blockchain technology. This review aims to provide new perspectives and a research foundation for the integrated development of blockchain technology and machine learning, supporting further innovation and application expansion of blockchain technology.
AfterLearnER (After Learning Evolutionary Retrofitting) consists in applying non-differentiable optimization, including evolutionary methods, to refine fully-trained machine learning models by optimizing a set of carefully chosen parameters or hyperparameters of the model, with respect to some actual, exact, and hence possibly non-differentiable error signal, performed on a subset of the standard validation set. The efficiency of AfterLearnER is demonstrated by tackling non-differentiable signals such as threshold-based criteria in depth sensing, the word error rate in speech re-synthesis, image quality in 3D generative adversarial networks (GANs), image generation via Latent Diffusion Models (LDM), the number of kills per life at Doom, computational accuracy or BLEU in code translation, and human appreciations in image synthesis. In some cases, this retrofitting is performed dynamically at inference time by taking into account user inputs. The advantages of AfterLearnER are its versatility (no gradient is needed), the possibility to use non-differentiable feedback including human evaluations, the limited overfitting, supported by a theoretical study and its anytime behavior. Last but not least, AfterLearnER requires only a minimal amount of feedback, i.e., a few dozens to a few hundreds of scalars, rather than the tens of thousands needed in most related published works. Compared to fine-tuning (typically using the same loss, and gradient-based optimization on a smaller but still big dataset at a fine grain), AfterLearnER uses a minimum amount of data on the real objective function without requiring differentiability.
Despite the significant advancements, existing object removal methods struggle with incomplete removal, incorrect content synthesis and blurry synthesized regions, resulting in low success rates. Such issues are mainly caused by the lack of high-quality paired training data, as well as the self-supervised training paradigm adopted in these methods, which forces the model to in-paint the masked regions, leading to ambiguity between synthesizing the masked objects and restoring the background. To address these issues, we propose a semi-supervised learning strategy with human-in-the-loop to create high-quality paired training data, aiming to train a Robust Object Remover (RORem). We first collect 60K training pairs from open-source datasets to train an initial object removal model for generating removal samples, and then utilize human feedback to select a set of high-quality object removal pairs, with which we train a discriminator to automate the following training data generation process. By iterating this process for several rounds, we finally obtain a substantial object removal dataset with over 200K pairs. Fine-tuning the pre-trained stable diffusion model with this dataset, we obtain our RORem, which demonstrates state-of-the-art object removal performance in terms of both reliability and image quality. Particularly, RORem improves the object removal success rate over previous methods by more than 18\%. The dataset, source code and trained model are available at https://github.com/leeruibin/RORem.
The massive and large-scale design of foundational semiconductor integrated circuits (ICs) is crucial to sustaining the advancement of many emerging and future technologies, such as generative AI, 5G/6G, and quantum computing. Excitingly, recent studies have shown the great capabilities of foundational models in expediting the design of digital ICs. Yet, applying generative AI techniques to accelerate the design of analog ICs remains a significant challenge due to critical domain-specific issues, such as the lack of a comprehensive dataset and effective representation methods for analog circuits. This paper proposes, $\textbf{AnalogGenie}$, a $\underline{\textbf{Gen}}$erat$\underline{\textbf{i}}$ve $\underline{\textbf{e}}$ngine for automatic design/discovery of $\underline{\textbf{Analog}}$ circuit topologies--the most challenging and creative task in the conventional manual design flow of analog ICs. AnalogGenie addresses two key gaps in the field: building a foundational comprehensive dataset of analog circuit topology and developing a scalable sequence-based graph representation universal to analog circuits. Experimental results show the remarkable generation performance of AnalogGenie in broadening the variety of analog ICs, increasing the number of devices within a single design, and discovering unseen circuit topologies far beyond any prior arts. Our work paves the way to transform the longstanding time-consuming manual design flow of analog ICs to an automatic and massive manner powered by generative AI. Our source code is available at https://github.com/xz-group/AnalogGenie.
Localization is one of the core parts of modern robotics. Classic localization methods typically follow the retrieve-then-register paradigm, achieving remarkable success. Recently, the emergence of end-to-end localization approaches has offered distinct advantages, including a streamlined system architecture and the elimination of the need to store extensive map data. Although these methods have demonstrated promising results, current end-to-end localization approaches still face limitations in robustness and accuracy. Bird's-Eye-View (BEV) image is one of the most widely adopted data representations in autonomous driving. It significantly reduces data complexity while preserving spatial structure and scale consistency, making it an ideal representation for localization tasks. However, research on BEV-based end-to-end localization remains notably insufficient. To fill this gap, we propose BEVDiffLoc, a novel framework that formulates LiDAR localization as a conditional generation of poses. Leveraging the properties of BEV, we first introduce a specific data augmentation method to significantly enhance the diversity of input data. Then, the Maximum Feature Aggregation Module and Vision Transformer are employed to learn robust features while maintaining robustness against significant rotational view variations. Finally, we incorporate a diffusion model that iteratively refines the learned features to recover the absolute pose. Extensive experiments on the Oxford Radar RobotCar and NCLT datasets demonstrate that BEVDiffLoc outperforms the baseline methods. Our code is available at https://github.com/nubot-nudt/BEVDiffLoc.
비정형 업무·규제 문서에서 조건 분기와 병렬 실행을 포함한 비즈니스 규칙 흐름을 구조화하는 Logic Gap을 해소하기 위해 30개 이상 도메인의 409개 문서와 2,855개 전문가 주석 규칙으로 구성된 BREX 벤치마크와 구조 인식 추론 프레임워크 ExIde를 제안한다. 13개 대규모 언어 모델(LLM) 평가에서 실행 가능한 그라운딩(executable grounding) 의사코드 생성이 일반 프롬프트보다 규칙 추출에 우수했고, 추론 최적화 모델이 장거리·비선형 규칙 의존성 추적에서 더 나은 성능을 보였다.
While virtual try-on (VTON) systems aim to render a garment onto a target person image, this paper tackles the novel task of virtual try-off (VTOFF), which addresses the inverse problem: generating standardized product images of garments from real-world photos of clothed individuals. Unlike VTON, which must resolve diverse pose and style variations, VTOFF benefits from a consistent and well-defined output format -- typically a flat, lay-down-style representation of the garment -- making it a promising tool for data generation and dataset enhancement. However, existing VTOFF approaches face two major limitations: (i) difficulty in disentangling garment features from occlusions and complex poses, often leading to visual artifacts, and (ii) restricted applicability to single-category garments (e.g., upper-body clothes only), limiting generalization. To address these challenges, we present Text-Enhanced MUlti-category Virtual Try-Off (TEMU-VTOFF), a novel architecture featuring a dual DiT-based backbone with a modified multimodal attention mechanism for robust garment feature extraction. Our architecture is designed to receive garment information from multiple modalities like images, text, and masks to work in a multi-category setting. Finally, we propose an additional alignment module to further refine the generated visual details. Experiments on VITON-HD and Dress Code datasets show that TEMU-VTOFF sets a new state-of-the-art on the VTOFF task, significantly improving both visual quality and fidelity to the target garments.
Diffusion-based image generation models excel at producing high-quality synthetic content, but suffer from slow and computationally expensive inference. Prior work has attempted to mitigate this by caching and reusing features within diffusion transformers across inference steps. These methods, however, often rely on rigid heuristics that result in limited acceleration or poor generalization across architectures. We propose Evolutionary Caching to Accelerate Diffusion models (ECAD), a genetic algorithm that learns efficient, per-model, caching schedules forming a Pareto frontier, using only a small set of calibration prompts. ECAD requires no modifications to network parameters or reference images. It offers significant inference speedups, enables fine-grained control over the quality-latency trade-off, and adapts seamlessly to different diffusion models. Notably, ECAD's learned schedules can generalize effectively to resolutions and model variants not seen during calibration. We evaluate ECAD on PixArt-alpha, PixArt-Sigma, and FLUX-1.dev using multiple metrics (FID, CLIP, Image Reward) across diverse benchmarks (COCO, MJHQ-30k, PartiPrompts), demonstrating consistent improvements over previous approaches. On PixArt-alpha, ECAD identifies a schedule that outperforms the previous state-of-the-art method by 4.47 COCO FID while increasing inference speedup from 2.35x to 2.58x. Our results establish ECAD as a scalable and generalizable approach for accelerating diffusion inference. Our project website is available at https://aniaggarwal.github.io/ecad and our code is available at https://github.com/aniaggarwal/ecad.
Remote sensing scene classification (RSSC) is traditionally formulated as a vision-centric problem, where inference relies on rich visual representations extracted from images. In this work, we revisit this assumption by introducing text-only RSSC, a new problem setting that investigates whether remote sensing scenes can be recognized using only textual semantic representations. To enable this study, we propose SeProRS, which reformulates RSSC as a text-only classification task through semantic proxy generation. Specifically, a vision-language model (VLM) is first employed to convert remote sensing images into textual descriptions, termed semantic proxies, which are then encoded by a text embedding model for scene classification. This decoupled pipeline provides a unified and reproducible benchmark for evaluating text-only scene understanding. Furthermore, we investigate representative prompting strategies and proxy generators, and analyze their impact on classification performance, text embedding behavior, cross-generator transferability, and efficiency. Extensive experiments on three well-known RSSC datasets show that semantic proxies can preserve substantial discriminative information, while also revealing inherent limitations of purely text-based scene understanding. This paper does not aim to replace image-based RSSC, but instead establishes a new research direction by quantifying the feasibility and boundaries of semantic proxy-based scene recognition. The proposed benchmark and analysis provide a foundation for future research on efficient and modality-reduced remote sensing scene understanding. The source code and datasets will be accessible at https://github.com/sstary/SSRS.
비정형·노이즈가 많은 사용자 상호작용 로그에서 유용한 피드백을 구분하기 어렵고 오프-정책 최적화 문제가 있다는 한계를 해결하기 위해, UNO(User log-driveN Optimization)는 로그를 반정형 규칙과 선호 쌍으로 증류하고 쿼리·피드백 기반 클러스터링과 인지적 격차 평가로 1차적·성찰적 경험 모듈을 구성해 LLM 시스템을 개선한다. MemoryBench 등에서 RAG 및 메모리 기반 베이스라인을 크게 능가하는 최첨단 수준의 효과성과 효율성을 보였다.
도메인 간 정책 전이는 소스 도메인과 타겟 도메인 간의 동역학 불일치로 인해 강화학습에서 중요한 과제이다. 본 논문에서는 정책이 충분한 데이터로 소스 도메인에서 학습되고, 타겟 도메인과의 상호작용은 제한적으로만 허용되는 온라인 동역학 적응(online dynamics adaptation) 설정을 고려한다. 동역학 불일치를 도메인 분류기(domain classifier), 가치 유도 데이터 필터링(value-guided data filtering), 또는 표현 학습(representation learning)을 활용해 해결하는 기존 연구들이 소수 존재한다. 대신, 우리는 생성 모델링(generative modeling) 관점에서 도메인 적응 문제를 연구한다. 구체적으로, 다음 상태 생성 과정에서 소스와 타겟 도메인 생성 궤적(generative trajectory) 간의 불일치를 활용해 동역학 불일치를 추정하는 확산 기반 프레임워크 DADiff를 제안한다. 타겟 도메인에 정책을 적응시키기 위해 보상 수정과 데이터 선택 변형을 모두 개발한다. 또한 이론적 분석을 제공하여, 주어진 정책의 두 도메인 간 성능 차이가 생성 궤적 편차(generative trajectory deviation)에 의해 상한이 정해짐을 보인다. 변형들의 적용 가능성과 우리의 이론적 분석과 선행 연구 간의 연결에 대한 추가 논의도 제공한다. 다양한 시프트가 있는 환경에서 광범위한 실험을 수행하여 우리 방법의 효과를 검증한다. 결과는 우리 방법이 기존 접근법 대비 우수한 성능을 제공하며 동역학 불일치를 효과적으로 해결함을 보여준다. 우리 방법의 코드는 https://github.com/hanyang-chen/DADiff-release 에서 제공한다.
에이전트 대규모 언어 모델(Agentic LLM)은 외부 도구 결과에 숨겨진 악의적인 보조 작업과 같은 간접 프롬프트 주입(IPI) 공격에 취약합니다. 이러한 위협을 해결하기 위한 많은 노력이 있었지만, IPI 공격에 노출되었을 때(이를 IPI 노출 상태라 명명) 에이전트 LLM의 내부 동작에 대해서는 거의 알려져 있지 않습니다. 본 논문에서는 이 문제를 세 가지 측면에서 심층 연구합니다. (1) 프로빙(Probing): 753B 매개변수를 가진 거대 모델 GLM-5.2를 포함한 6개 모델 전반에서, 생성 이전 은닉 상태(hidden states)로 학습된 단순 선형 프로브가 LLM의 IPI 노출을 예측할 수 있음을 확인했습니다. 이 프로브들은 보지 못한(unseen) 공격, 에이전트 지시문, 작업 모음 전반에서 90% 이상의 AUROC를 달성했으며, 적응형 공격과 교차 언어 환경에서도 높은 강건성을 보입니다. (2) 방어(Defense): 사고의 연쇄(CoT) 측정 결과 '인식-행동 격차'가 드러났습니다. 즉, 모델이 이러한 신호를 인코딩함에도 불구하고 이를 안전한 행동으로 전환하지 못하는 경우가 많았습니다. 이에 따라 필요 시 주입 방지 추론을 앞에 덧붙이는 프로브 게이트 기반 추론 방어 기법인 AGRI를 제안합니다. 까다로운 AgentDojo 환경에서 AGRI는 정상 작업의 유용성을 대체로 유지하면서 공격 성공률을 대폭 감소시켰으며, 일례로 Qwen3.5-27B에서 34.6%에서 0%로 줄였습니다. (3) 설명(Explanation): 프로브가 포착한 신호와 가장 강하게 상관관계를 보이는 자연어 설명을 식별하는 분석 프레임워크를 도입합니다. 도출된 프로파일은 모델마다 차이를 보이며, 잠재 신호는 직접적인 IPI 노출 주장 또는 간접적인 운영 단서와 정렬될 수 있습니다. 코드는 https://github.com/jianshuod/IPI-exposure-signal 에서 확인할 수 있습니다.
비디오 가상 착용(VVT) 기술은 상당한 발전을 이루었으나, 기존 방법들은 여전히 두 가지 근본적인 한계를 보입니다. 첫째, 단일 의류 변환에 국한되어 여러 아이템을 동시에 착용해보는 것이 매우 비현실적입니다. 둘째, 명시적인 외부 사전 정보(예: 의류 마스크)에 크게 의존하여 중요한 물리적 동역학을 불가피하게 훼손하고 시각적 품질을 저하시킵니다. 이러한 격차를 해소하기 위해, 본 논문에서는 단 한 번의 추론(inference) 과정으로 비디오 속 인물에게 다양한 착용 가능한 객체를 동시에 변환하는 것을 목표로 하는 새로운 Try-On Anything 태스크를 제안합니다. 이 패러다임을 지원하고 표준화하기 위해 맞춤형 평가 프로토콜과 함께 쌍을 이룬 비디오 데이터셋을 포함하는 포괄적인 벤치마크인 TryAny-Bench를 도입합니다. 나아가, 이 태스크를 해결하기 위해 외부 사전 정보가 필요 없는(external-prior-free) 생성 프레임워크인 OmniTryOn을 제시합니다. 구체적으로 OmniTryOn은 초기 비디오 프레임을 통해 생성 과정에 다양한 착용 객체를 직접 제공하는 First Frame Wearable Cache 전략을 사용합니다. 일관성을 유지하기 위해 우리는 복잡한 인체 움직임과 배경 동역학을 엄격하게 보존하는 강력한 시공간 앵커를 본질적으로 구축하는 시공간 일관성 RoPE(STC-RoPE)를 제안합니다. 제안된 점진적 가상 착용(GTO) 학습(training) 전략으로 최적화된 본 모델은 점진적으로 강력한 다중 객체 합성을 숙달합니다. TryAny-Bench에 대한 광범위한 실험을 통해 OmniTryOn이 기존의 특화된 비디오 가상 착용 모델들과 일반적인 비디오 편집 베이스라인들을 크게 능가하여 Try-On Anything 태스크에 대한 강력하고 새로운 기준을 확립함을 입증합니다. 데이터셋, 코드 및 모델은 https://github.com/xcltql666/OminTryOn 에서 확인할 수 있습니다.
AI 코딩 에이전트 벤치마크는 Chen et al. (2021)의 pass@k 추정량으로 에이전트 순위를 매기지만, 현재 구현은 이를 잘못 적용한다. 단일 제출물의 단위 테스트 개수를 n으로 두는 반면, n은 독립적인 롤아웃 시도 횟수여야 하며, 테스트 스위트 크기와 시도 독립성을 혼동한다. 우리는 이 운영화 오류를 진단하고 반례로 증명한 뒤, 동일 추정량을 올바르게 적용한 reliability@k를 제안한다. 여기서 n은 독립 롤아웃 수이고, c는 (과제, 에이전트) 쌍당 완전 통과 롤아웃 수이다. 합성 다중 롤아웃 벤치마크에서 잘못 적용된 지표는 보고 점수를 절대값 기준 0.85–0.97만큼 부풀리며(보고 0.96–0.98 대 보정 0.00–0.12), 값싼 단일 롤아웃 대리 지표는 반복 실행을 대체하지 못한다(Spearman ρ= 0.417). 기능적 정확성이 보안상 안전(safety)을 함의하지 않는다는 증거에 기인하여, 우리는 추가로 security-adjusted reliability@k를 제안한다. 이는 기능적으로 올바르면서 고심각도 비안전 패턴이 없는 롤아웃만 집계한다. 세 에이전트에 대한 초기 live-API 테스트에서, 현재 스캐너와 임계값 하에서는 이 조정이 순위를 바꾸지 않았으므로, 우리는 이를 보완적 관점으로 제시하며, 결정적 평가는 표본력이 더 큰 후속 실험이 필요하다고 밝힌다. 마지막으로, 5개 과제 SWE-bench Verified 예비 파일럿은 실제 저장소 설정에서 동일한 핵심 우려를 관찰한다. 매크로 평균 hidden-test 통과율은 0.80이었고 엄격한 과제 해결률은 0.20이었다.
자율주행과 같은 애플리케이션의 발전과 함께 객체 검출(object detection)이 상당한 주목을 받고 있으나, 동시에 모델의 무결성을 심각하게 훼손하는 백도어 공격과 같은 치명적인 취약점도 부각되고 있다. 구체적으로 이러한 공격은 입력에 사전 정의된 트리거(trigger)가 존재할 때 객체의 범주를 변경하거나(즉, 객체 오분류), 경계 상자(bounding box)를 제거하거나(즉, 객체 소멸), 존재하지 않는 객체에 대한 경계 상자 제안(proposal)을 생성(즉, 객체 생성)하는 방식을 취한다. 이미지 분류(image classification)에 대한 백도어 방어는 잘 확립되어 있지만, 객체 검출에 대한 연구는 상대적으로 미진한 상태이다. 기존 방어 기법들은 잠재적인 백도어를 찾기 위해 출력이나 모델을 스캔하여 이러한 위협을 해결하지만, 악성 데이터나 모델 자체를 폐기해야 한다. 이러한 해결책은 객체 검출 파이프라인에서 지속적이고 정확한 인지 스트림을 제공하지 못한다. 이러한 한계를 해결하기 위해, 본 논문에서는 안정적인 인지 흐름을 보장하는 입력 정제(input purification)에 기반한 객체 검출용 새로운 입력 단계 블랙박스 방어 기법인 ODPure를 제안한다. 객체 검출기의 밀집 예측(dense prediction) 특성에 맞춰 설계된 제안하는 손상-재구성-선택(Corruption-Reconstruction-Selection, CRS) 패러다임은 다양한 손상(corruption) 포트폴리오를 통해 트리거를 무력화하여 방대한 중복 제안 풀을 생성하고, 생성형 사전 확률(generative priors)을 통해 세밀한 구조적 단서를 복원한 뒤, 최종적으로 투표를 적용하여 도출된 검출 결과에 대한 합의에 도달하는 방식으로 작동한다. 포괄적인 실험을 통해 제안 기법이 기준선(baseline) 정확도를 보존하면서 다양한 백도어 공격 및 트리거 유형에 대해 강력한 방어를 제공함을 입증한다. 코드는 https://github.com/Alex66366/ODPure 에서 확인할 수 있다.