VLMAE: Vision-Language Masked Autoencoder
원문 초록
Image and language modeling is of crucial importance for vision-language pre-training (VLP), which aims to learn multi-modal representations from large-scale paired image-text data. However, we observe that most existing VLP methods focus on modeling the interactions between image and text features while neglecting the information disparity between image and text, thus suffering from focal bias. To address this problem, we propose a vision-language masked autoencoder framework (VLMAE). VLMAE employs visual generative learning, facilitating the model to acquire fine-grained and unbiased features. Unlike the previous works, VLMAE pays attention to almost all critical patches in an image, providing more comprehensive understanding. Extensive experiments demonstrate that VLMAE achieves better performance in various vision-language downstream tasks, including visual question answering, image-text retrieval and visual grounding, even with up to 20% pre-training speedup.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
From RAG to Agentic: Validating Islamic-Medicine Responses with LLM Agents
Describe Anything Model for Visual Question Answering on Text-rich Images
Kodezi Chronos: A Debugging-First Language Model for Repository-Scale, Memory-Driven Code Understanding
Differential Mamba
RMIT-ADM+S at the SIGIR 2025 LiveRAG Challenge
Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility
토론
이 논문과 연결된 포럼 글이 아직 없습니다.