Papers with Code · CC BY-SA 4.0

Image-to-Text Retrieval

33 평가 기록 8 datasets

Image-text retrieval is the process of retrieving relevant images based on textual descriptions or finding corresponding textual descriptions for a given image. This task is interdisciplinary, combining techniques from computer vision, and natural language processing. The primary challenge lies in bridging the semantic gap — the difference between how visual data is represented in images and how humans describe that information using language. To address this, many methods focus on learning a shared embedding space where both images and text can be represented in a comparable way, allowing their similarities to be measured and facilitating more accurate retrieval.

Source: <span class="description-source"> Extending CLIP for Category-to-Image Retrieval in E-commerce</span>

개요는 각 항목의 일부입니다. 위 탭에서 전체 기록을 페이지별로 탐색할 수 있습니다.

벤치마크

Papers with Code 아카이브의 데이터셋별 기록 예시입니다. 평가 조건과 지표 방향이 검증되지 않아 순위를 의미하지 않습니다.

Dataset기록된 모델 예시MetricPaperCode
InternVL-G-FT (finetuned, w/o ranking)
Recall@1: 97.9Recall@5: 100Recall@10: 100
논문 코드
BLIP-2 (ViT-G, fine-tuned)
Recall@10: 98.5Recall@1: 85.4Recall@5: 97.0
논문 코드
BLIP2 FlanT5-XXL (Text-only FT)
Specificity: 94
논문 -
ERNIE-ViL2.0
Recall@1: 33.7Recall@5: 52.1Recall@10: 60.0
논문 코드
SigLIP (ViT-L, zero-shot)
Recall@1: 70.6
논문 코드
FETA's CLIP-MIL (Many-Shot Image-to-text)
R@1: 35.5R@5: 58.3R@10: 67
논문 코드
GeoRSCLIP-FT
Image to Text Recall@1: 22.14%
논문 코드
CMCL
Recall@1: 36.1Recall@5: 55.5Recall@10: 62.2
논문 코드

데이터셋

8개

논문

라이브러리

Papers with Code 리더보드 row에 연결된 구현 저장소입니다.