Cross-Modal Information Retrieval
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 1
- 평가표
- 0
- 보존된 평가 기록
- 0
Cross-Modal Information Retrieval (CMIR) is the task of finding relevant items across different modalities. For example, given an image, find a text or vice versa. The main challenge in CMIR is known as the heterogeneity gap: since items from different modalities have different data types, the similarity between them cannot be measured directly. Therefore, the majority of CMIR methods published to date attempt to bridge this gap by learning a latent representation space, where the similarity between items from different modalities can be measured.
Source: Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching Models
Rethinking Benchmarks for Cross-modal Image-text Retrieval
Image-text Retrieval via Preserving Main Semantics of Vision
Is Cross-modal Information Retrieval Possible without Training?
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset
CoVLR: Coordinating Cross-Modal Consistency and Intra-Modal Structure for Vision-Language Retrieval
Noisy Correspondence Learning with Meta Similarity Correction
Exposing and Mitigating Spurious Correlations for Cross-Modal Retrieval
AToMiC: An Image/Text Retrieval Test Collection to Support Multimedia Content Creation
Multi-Modal Representation Learning with Text-Driven Soft Masks
Hindi as a Second Language: Improving Visually Grounded Speech with Semantically Similar Samples
MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks
Multimodal Image-Text Matching Improves Retrieval-based Chest X-Ray Report Generation
Integrating Language Guidance Into Image-Text Matching for Correcting False Negatives
Increasing Textual Context Size Boosts Medical Image-Text Matching
Plug-and-Play Regulators for Image-Text Matching
BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistency
Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval
MXM-CLR: A Unified Framework for Contrastive Learning of Multifold Cross-Modal Representations
Single-branch Network for Multimodal Training
Refined Vision-Language Modeling for Fine-grained Multi-modal Pre-training
Adversarial Modality Alignment Network for Cross-Modal Molecule Retrieval
Cross-modal Retrieval with Improved Graph Convolution
FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks
Selectively Hard Negative Mining for Alleviating Gradient Vanishing in Image-Text Matching
BrainCLIP: Bridging Brain and Visual-Linguistic Representation Via CLIP for Generic Natural Visual Stimulus Decoding
Data leakage in cross-modal retrieval training: A case study
X-TRA: Improving Chest X-ray Tasks with Cross-Modal Retrieval Augmentation
Cross-Modal Retrieval with Partially Mismatched Pairs
Multi-modal Machine Learning in Engineering Design: A Review and Future Directions
VITR: Augmenting Vision Transformers with Relation-Focused Learning for Cross-Modal Information Retrieval
Distribution Aligned Feature Clustering for Zero-Shot Sketch-Based Image Retrieval
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks
Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study
Pix2Map: Cross-modal Retrieval for Inferring Street Maps from Images
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.