Suspected Object Matters: Rethinking Model's Prediction for One-stage Visual Grounding
원문 초록
Recently, one-stage visual grounders attract high attention due to their comparable accuracy but significantly higher efficiency than two-stage grounders. However, inter-object relation modeling has not been well studied for one-stage grounders. Inter-object relationship modeling, though important, is not necessarily performed among all objects, as only part of them are related to the text query and may confuse the model. We call these objects suspected objects. However, exploring their relationships in the one-stage paradigm is non-trivial because: First, no object proposals are available as the basis on which to select suspected objects and perform relationship modeling. Second, suspected objects are more confusing than others, as they may share similar semantics, be entangled with certain relationships, etc, and thereby more easily mislead the model prediction. Toward this end, we propose a Suspected Object Transformation mechanism (SOT), which can be seamlessly integrated into existing CNN and Transformer-based one-stage visual grounders to encourage the target object selection among the suspected ones. Suspected objects are dynamically discovered from a learned activation map adapted to the model current discrimination ability during training. Afterward, on top of suspected objects, a Keyword-Aware Discrimination module (KAD) and an Exploration by Random Connection strategy (ERC) are concurrently proposed to help the model rethink its initial prediction. On the one hand, KAD leverages keywords contributing high to suspected object discrimination. On the other hand, ERC allows the model to seek the correct object instead of being trapped in a situation that always exploits the current false prediction. Extensive experiments demonstrate the effectiveness of our proposed method.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World
InstructSAM: A Training-Free Framework for Instruction-Oriented Remote Sensing Object Recognition
From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes
RoHOI: Robustness Benchmark for Human-Object Interaction Detection
Car Object Counting and Position Estimation via Extension of the CLIP-EBC Framework
GTA1: GUI Test-time Scaling Agent
토론
이 논문과 연결된 포럼 글이 아직 없습니다.