Cross-Modal Information Retrieval
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 1
- 평가표
- 0
- 보존된 평가 기록
- 0
Cross-Modal Information Retrieval (CMIR) is the task of finding relevant items across different modalities. For example, given an image, find a text or vice versa. The main challenge in CMIR is known as the heterogeneity gap: since items from different modalities have different data types, the similarity between them cannot be measured directly. Therefore, the majority of CMIR methods published to date attempt to bridge this gap by learning a latent representation space, where the similarity between items from different modalities can be measured.
Source: Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
Evaluating Attribute Comprehension in Large Vision-Language Models
Towards Deconfounded Image-Text Matching with Causal Inference
Leveraging Chemistry Foundation Models to Facilitate Structure Focused Retrieval Augmented Generation in Multi-Agent Workflows for Catalyst and Materials Design
Limitations in Employing Natural Language Supervision for Sensor-Based Human Activity Recognition -- And Ways to Overcome Them
Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach
Dynamic and Compressive Adaptation of Transformers From Images to Videos
Contrastive masked auto-encoders based self-supervised hashing for 2D image and 3D point cloud cross-modal retrieval
Efficient and Versatile Robust Fine-Tuning of Zero-shot Models
Disentangled Noisy Correspondence Learning
PC$^2$: Pseudo-Classification Based Pseudo-Captioning for Noisy Correspondence Learning in Cross-Modal Retrieval
Image-text matching for large-scale book collections
Start from Video-Music Retrieval: An Inter-Intra Modal Loss for Cross Modal Retrieval
DAC: 2D-3D Retrieval with Noisy Labels via Divide-and-Conquer Alignment and Correction
Unified Lexical Representation for Interpretable Visual-Language Alignment
Revolutionizing Text-to-Image Retrieval as Autoregressive Token-to-Voken Generation
Aligning Sight and Sound: Advanced Sound Source Localization Through Audio-Visual Alignment
ModalChorus: Visual Probing and Alignment of Multi-modal Embeddings via Modal Fusion Map
UGNCL: Uncertainty-Guided Noisy Correspondence Learning for Efficient Cross-Modal Matching
Second Place Solution of WSDM2023 Toloka Visual Question Answering Challenge
Semantic Compositions Enhance Vision-Language Contrastive Learning
Improving the Consistency in Cross-Lingual Cross-Modal Retrieval with 1-to-K Contrastive Learning
MATE: Meet At The Embedding -- Connecting Images with Long Texts
ACE: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling
Efficient and Long-Tailed Generalization for Pre-trained Vision-Language Model
Generative Visual Instruction Tuning
Composing Object Relations and Attributes for Image-Text Matching
Deep Sketched Output Kernel Regression for Structured Prediction
Advanced Multimodal Deep Learning Architecture for Image-Text Matching
What If We Recaption Billions of Web Images with LLaMA-3?
Merlin: A Vision Language Foundation Model for 3D Computed Tomography
Separating the "Chirp" from the "Chat": Self-supervised Visual Grounding of Sound and Language
Hire: Hybrid-modal Interaction with Multiple Relational Enhancements for Image-Text Matching
No Captions, No Problem: Captionless 3D-CLIP Alignment with Hard Negatives via CLIP Knowledge and LLMs
CaLa: Complementary Association Learning for Augmenting Composed Image Retrieval
Multi-Modal Generative Embedding Model
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.