Papers with Code 아카이브

Image-to-Text Retrieval

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크: 0
평가표: 8
보존된 평가 기록: 33

Image-text retrieval is the process of retrieving relevant images based on textual descriptions or finding corresponding textual descriptions for a given image. This task is interdisciplinary, combining techniques from computer vision, and natural language processing. The primary challenge lies in bridging the semantic gap — the difference between how visual data is represented in images and how humans describe that information using language. To address this, many methods focus on learning a shared embedding space where both images and text can be represented in a comparable way, allowing their similarities to be measured and facilitating more accurate retrieval.

Source: Extending CLIP for Category-to-Image Retrieval in E-commerce

논문

이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.

Image-to-Text Retrieval

논문

Improving Medical Visual Representation Learning with Pathological-level Cross-Modal Alignment and Correlation Exploration

Efficient Medical Vision-Language Alignment Through Adapting Masked Vision Models

Towards Cross-modal Retrieval in Chinese Cultural Heritage Documents: Dataset and Solution

SemCORE: A Semantic-Enhanced Generative Cross-Modal Retrieval Framework with MLLMs

DART: Disease-aware Image-Text Alignment and Self-correcting Re-alignment for Trustworthy Radiology Report Generation

ABC: Achieving Better Control of Multimodal Embeddings using VLMs

DIR: Retrieval-Augmented Image Captioning with Comprehensive Understanding

Robotic State Recognition with Image-to-Text Retrieval Task of Pre-Trained Vision-Language Model and Black-Box Optimization

Robotic Environmental State Recognition with Pre-Trained Vision-Language Models and Black-Box Optimization

GABInsight: Exploring Gender-Activity Binding Bias in Vision-Language Models

Towards a text-based quantitative and explainable histopathology image analysis

BiVLC: Extending Vision-Language Compositionality Evaluation with Text-to-Image Retrieval

Understanding the Effect of using Semantically Meaningful Tokens for Visual Representation Learning

Efficient Remote Sensing with Harmonized Transfer Learning and Modality Alignment

CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?

InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks

Negative Pre-aware for Noisy Cross-modal Matching

Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval

Vision-Language Dataset Distillation

PRIOR: Prototype Representation Joint Learning from Medical Images and Reports

Towards a Visual-Language Foundation Model for Computational Pathology

RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing

CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers

ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities

Rethinking Benchmarks for Cross-modal Image-text Retrieval

Is Cross-modal Information Retrieval Possible without Training?

Sigmoid Loss for Language Image Pre-Training

Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images

UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers

BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

HADA: A Graph-based Amalgamation Framework in Image-text Retrieval

When are Lemons Purple? The Concept Association Bias of Vision-Language Models

A survey on knowledge-enhanced multimodal learning

AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities

ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training

FETA: Towards Specializing Foundation Models for Expert Task Applications