Improving Referring Expression Grounding with Cross-modal Attention-guided Erasing
원문 초록
Referring expression grounding aims at locating certain objects or persons in an image with a referring expression, where the key challenge is to comprehend and align various types of information from visual and textual domain, such as visual attributes, location and interactions with surrounding regions. Although the attention mechanism has been successfully applied for cross-modal alignments, previous attention models focus on only the most dominant features of both modalities, and neglect the fact that there could be multiple comprehensive textual-visual correspondences between images and referring expressions. To tackle this issue, we design a novel cross-modal attention-guided erasing approach, where we discard the most dominant information from either textual or visual domains to generate difficult training samples online, and to drive the model to discover complementary textual-visual correspondences. Extensive experiments demonstrate the effectiveness of our proposed method, which achieves state-of-the-art performance on three referring expression grounding datasets.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
What is the Right Embedding Space for Contrastive Learning in REC?
Detecting Referring Expressions in Visually Grounded Dialogue with Autoregressive Language Models
TextRegion: Text-Aligned Region Tokens from Frozen Image-Text Models
RemoteSAM: Towards Segment Anything for Earth Observation
Vision-Language Models Are Not Pragmatically Competent in Referring Expression Generation
4th PVUW MeViS 3rd Place Report: Sa2VA
토론
이 논문과 연결된 포럼 글이 아직 없습니다.