Image Pivoting for Learning Multilingual Multimodal Representations
원문 초록
In this paper we propose a model to learn multimodal multilingual representations for matching images and sentences in different languages, with the aim of advancing multilingual versions of image search and image understanding. Our model learns a common representation for images and their descriptions in two different languages (which need not be parallel) by considering the image as a pivot between two languages. We introduce a new pairwise ranking loss function which can handle both symmetric and asymmetric similarity between the two modalities. We evaluate our models on image-description ranking for German and English, and on semantic textual similarity of image descriptions in English. In both cases we achieve state-of-the-art performance.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Temporally-Grounded Language Generation: Towards Real-Time Vision-Language Models
Text-Visual Semantic Constrained AI-Generated Image Quality Assessment
RadiomicsRetrieval: A Customizable Framework for Medical Image Retrieval Using Radiomics Features
MS-DPPs: Multi-Source Determinantal Point Processes for Contextual Diversity Refinement of Composite Attributes in Text to Image Retrieval
Orchestrator-Agent Trust: A Modular Agentic AI Visual Classification System with Trust-Aware Orchestration and RAG-Based Reasoning
FA: Forced Prompt Learning of Vision-Language Models for Out-of-Distribution Detection
토론
이 논문과 연결된 포럼 글이 아직 없습니다.