Cross-Modal Information Retrieval
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 1
- 평가표
- 0
- 보존된 평가 기록
- 0
Cross-Modal Information Retrieval (CMIR) is the task of finding relevant items across different modalities. For example, given an image, find a text or vice versa. The main challenge in CMIR is known as the heterogeneity gap: since items from different modalities have different data types, the similarity between them cannot be measured directly. Therefore, the majority of CMIR methods published to date attempt to bridge this gap by learning a latent representation space, where the similarity between items from different modalities can be measured.
Source: Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
Distinctive Image Captioning: Leveraging Ground Truth Captions in CLIP Guided Reinforcement Learning
Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond
Mind the Modality Gap: Towards a Remote Sensing Vision-Language Model via Cross-modal Alignment
Large Language Models for Captioning and Retrieving Remote Sensing Images
ColorSwap: A Color and Word Order Dataset for Multimodal Evaluation
Zero-shot sketch-based remote sensing image retrieval based on multi-level and attention-guided tokenization
MouSi: Poly-Visual-Expert Vision-Language Models
Cross-Modal Coordination Across a Diverse Set of Input Modalities
Beyond Image-Text Matching: Verb Understanding in Multimodal Transformers Using Guided Masking
Enhancing medical vision-language contrastive learning via inter-matching relation modelling
Enhancing Image-Text Matching with Adaptive Feature Aggregation
Developing ChatGPT for Biology and Medicine: A Complete Review of Biomedical Question Answering
Cross-modal Retrieval for Knowledge-based Visual Question Answering
Backdoor Attack on Unpaired Medical Image-Text Foundation Models: A Pilot Study on MedCLIP
Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation
Masked Contrastive Reconstruction for Cross-modal Medical Image-Report Retrieval
LeanVec: Searching vectors faster by making them fit
SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing
TF-CLIP: Learning Text-free CLIP for Video-based Person Re-Identification
CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfer
WikiMuTe: A web-sourced dataset of semantic descriptions for music audio
Negative Pre-aware for Noisy Cross-modal Matching
Enhancing Recipe Retrieval with Foundation Models: A Data Augmentation Perspective
OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization
Uni3DL: Unified Model for 3D and Language Understanding
CILF-CIAE: CLIP-driven Image-Language Fusion for Correcting Inverse Age Estimation
T3D: Advancing 3D Medical Vision-Language Pre-training by Learning Multi-View Visual Consistency
Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understanding
Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models
Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Models
Automatic Synthetic Data and Fine-grained Adaptive Feature Alignment for Composed Person Retrieval
Invisible Relevance Bias: Text-Image Retrieval Models Prefer AI-Generated Images
MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Experts
Contrastive Transformer Learning with Proximity Data Generation for Text-Based Person Search
Active Mining Sample Pair Semantics for Image-text Matching
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.