Semi-supervised Visual Feature Integration for Pre-trained Language Models
원문 초록
Integrating visual features has been proved useful for natural language understanding tasks. Nevertheless, in most existing multimodal language models, the alignment of visual and textual data is expensive. In this paper, we propose a novel semi-supervised visual integration framework for pre-trained language models. In the framework, the visual features are obtained through a visualization and fusion mechanism. The uniqueness includes: 1) the integration is conducted via a semi-supervised approach, which does not require aligned images for every sentences 2) the visual features are integrated as an external component and can be directly used by pre-trained language models. To verify the efficacy of the proposed framework, we conduct the experiments on both natural language inference and reading comprehension tasks. The results demonstrate that our mechanism brings improvement to two strong baseline models. Considering that our framework only requires an image database, and no not requires further alignments, it provides an efficient and feasible way for multimodal language learning.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
LogiQA 2.0—An Improved Dataset for Logical Reasoning in Natural Language Understanding
GATE: General Arabic Text Embedding for Enhanced Semantic Textual Similarity with Matryoshka Representation Learning and Hybrid Loss Training
Enhancing Text-to-Image Diffusion Transformer via Split-Text Conditioning
DIBERT: Dependency Injected Bidirectional Encoder Representations from Transformers
On Designing Diffusion Autoencoders for Efficient Generation and Representation Learning
Vision Language Action Models in Robotic Manipulation: A Systematic Review
토론
이 논문과 연결된 포럼 글이 아직 없습니다.