Cross-Modal Information Retrieval
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 1
- 평가표
- 0
- 보존된 평가 기록
- 0
Cross-Modal Information Retrieval (CMIR) is the task of finding relevant items across different modalities. For example, given an image, find a text or vice versa. The main challenge in CMIR is known as the heterogeneity gap: since items from different modalities have different data types, the similarity between them cannot be measured directly. Therefore, the majority of CMIR methods published to date attempt to bridge this gap by learning a latent representation space, where the similarity between items from different modalities can be measured.
Source: Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
Maybe you are looking for CroQS: Cross-modal Query Suggestion for Text-to-Image Retrieval
A Concept-Centric Approach to Multi-Modality Learning
Rebalanced Vision-Language Retrieval Considering Structure-Aware Distillation
ViUniT: Visual Unit Tests for More Robust Visual Programming
jina-clip-v2: Multilingual Multimodal Embeddings for Text and Images
CLIP-PING: Boosting Lightweight Vision-Language Models with Proximus Intrinsic Neighbors Guidance
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey
Fusing Physics-Driven Strategies and Cross-Modal Adversarial Learning: Toward Multi-Domain Applications
Multimodal Whole Slide Foundation Model for Pathology
Automatic Prompt Generation and Grounding Object Detection for Zero-Shot Image Anomaly Detection
VLM-HOI: Vision Language Models for Interpretable Human-Object Interaction Analysis
FLEX-CLIP: Feature-Level GEneration Network Enhanced CLIP for X-shot Cross-modal Retrieval
CLIPS: An Enhanced CLIP Framework for Learning with Synthetic Captions
FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularity
Everything is a Video: Unifying Modalities through Next-Frame Prediction
MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs
Exploring Optimal Transport-Based Multi-Grained Alignments for Text-Molecule Retrieval
TaxaBind: A Unified Embedding Space for Ecological Applications
Towards Cross-Modal Text-Molecule Retrieval with Better Modality Alignment
Nearest Neighbor Normalization Improves Multimodal Retrieval
Multilingual Vision-Language Pre-training for the Remote Sensing Domain
EntityCLIP: Entity-Centric Image-Text Matching via Multimodal Attentive Contrastive Learning
Bridging the Modality Gap: Dimension Information Alignment and Sparse Spatial Constraint for Image-Text Matching
Test-time Adaptation for Cross-modal Retrieval with Query Shift
Deep Class-guided Hashing for Multi-label Cross-modal Retrieval
GleanVec: Accelerating vector search with minimalist nonlinear dimensionality reduction
MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models
CSA: Data-efficient Mapping of Unimodal Features to Multimodal Features
BadCM: Invisible Backdoor Attack Against Cross-Modal Learning
Multimodal LLM Enhanced Cross-lingual Cross-modal Retrieval
Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation
DARE: Diverse Visual Question Answering with Robustness Evaluation
NEVLP: Noise-Robust Framework for Efficient Vision-Language Pre-training
EyeCLIP: A visual-language foundation model for multi-modal ophthalmic image analysis
M3-Jepa: Multimodal Alignment via Multi-directional MoE based on the JEPA framework
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.