Cross-Modal Information Retrieval
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 1
- 평가표
- 0
- 보존된 평가 기록
- 0
Cross-Modal Information Retrieval (CMIR) is the task of finding relevant items across different modalities. For example, given an image, find a text or vice versa. The main challenge in CMIR is known as the heterogeneity gap: since items from different modalities have different data types, the similarity between them cannot be measured directly. Therefore, the majority of CMIR methods published to date attempt to bridge this gap by learning a latent representation space, where the similarity between items from different modalities can be measured.
Source: Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
Aligning Information Capacity Between Vision and Language via Dense-to-Sparse Feature Distillation for Image-Text Matching
Adaptive Inner Speech-Text Alignment for LLM-based Speech Translation
NeighborRetr: Balancing Hub Centrality in Cross-Modal Retrieval
Astrea: A MOE-based Visual Understanding Model with Progressive Alignment
A Recipe for Improving Remote Sensing VLM Zero Shot Generalization
CLIP is Strong Enough to Fight Back: Test-time Counterattacks towards Zero-shot Adversarial Robustness of CLIP
X2CT-CLIP: Enable Multi-Abnormality Detection in Computed Tomography from Chest Radiography via Tri-Modal Contrastive Learning
Composed Multi-modal Retrieval: A Survey of Approaches and Applications
Recurrence-Enhanced Vision-and-Language Transformers for Robust Multimodal Document Retrieval
IteRPrimE: Zero-shot Referring Image Segmentation with Iterative Grad-CAM Refinement and Primary Word Emphasis
MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representations
Lightweight Contrastive Distilled Hashing for Online Cross-modal Retrieval
CLIP Under the Microscope: A Fine-Grained Analysis of Multi-Object Representation
ReCon: Enhancing True Correspondence Discrimination through Relation Consistency for Robust Noisy Correspondence Learning
On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation
Pathology Report Generation and Multimodal Representation Learning for Cutaneous Melanocytic Lesions
Object-centric Binding in Contrastive Language-Image Pretraining
CLASS: Enhancing Cross-Modal Text-Molecule Retrieval Performance and Training Efficiency
GAIA: A Global, Multi-modal, Multi-scale Vision-Language Dataset for Remote Sensing Image Analysis
Zero-Shot Interactive Text-to-Image Retrieval via Diffusion-Augmented Representations
MASS: Overcoming Language Bias in Image-Text Matching
TSVC:Tripartite Learning with Semantic Variation Consistency for Robust Image-Text Retrieval
FiLo++: Zero-/Few-Shot Anomaly Detection by Fused Fine-Grained Descriptions and Deformable Localization
Deep Reversible Consistency Learning for Cross-modal Retrieval
Robust Self-Paced Hashing for Cross-Modal Retrieval with Noisy Labels
Towards Identity-Aware Cross-Modal Retrieval: a Dataset and a Baseline
Multi-Head Attention Driven Dynamic Visual-Semantic Embedding for Enhanced Image-Text Matching
Dynamic Adapter with Semantics Disentangling for Cross-lingual Cross-modal Retrieval
Maybe you are looking for CroQS: Cross-modal Query Suggestion for Text-to-Image Retrieval
A Concept-Centric Approach to Multi-Modality Learning
Rebalanced Vision-Language Retrieval Considering Structure-Aware Distillation
ViUniT: Visual Unit Tests for More Robust Visual Programming
jina-clip-v2: Multilingual Multimodal Embeddings for Text and Images
CLIP-PING: Boosting Lightweight Vision-Language Models with Proximus Intrinsic Neighbors Guidance
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.