PK PyTorchKR Papers
Papers with Code · CC BY-SA 4.0

Image Captioning

448 평가 기록 42 datasets

Image Captioning is the task of describing the content of an image in words. This task lies at the intersection of computer vision and natural language processing. Most image captioning systems use an encoder-decoder framework, where an input image is encoded into an intermediate representation of the information in the image, and then decoded into a descriptive text sequence. The most popular benchmarks are nocaps and COCO, and models are typically evaluated according to a BLEU or CIDER metric.

( Image credit: Reflective Decoding Network for Image Captioning, ICCV'19)

개요는 각 항목의 일부입니다. 위 탭에서 전체 기록을 페이지별로 탐색할 수 있습니다.

벤치마크

Papers with Code 아카이브의 데이터셋별 기록 예시입니다. 평가 조건과 지표 방향이 검증되지 않아 순위를 의미하지 않습니다.

Dataset기록된 모델 예시MetricPaperCode
IBM Research AI
CIDEr: 80.67B1: 72.48B2: 53.93B3: 38.79
- -
mPLUG
BLEU-4: 46.5METEOR: 32.0CIDER: 155.1SPICE: 26.0
논문 코드
PaLI
CIDEr: 121.09B1: 88.02B2: 75.21B3: 59.38
논문 코드
PaLI
CIDEr: 126.67B1: 86.28B2: 71.19B3: 52.63
논문 코드
Lyrics
CIDEr: 126.8
논문 -
GIT2, Single Model
CIDEr: 125.51B1: 88.9B2: 75.86B3: 58.9
논문 코드
ExpansionNet v2
CIDEr: 143.7
논문 코드
IBM Research AI
CIDEr: 81.04B1: 72.77B2: 54.17B3: 38.97
- -
3D CoCa
CIDEr: 85.42BLEU-4: 45.56METEOR: 30.95ROUGE-L: 61.98
논문 코드
GIT2
CIDEr: 124.77B1: 88.43B2: 75.02B3: 57.87
논문 코드
TAP
CIDEr: 103.22B4: 21.86ROUGE-L: 45.65METEOR: 21.85
- -
GIT2
CIDEr: 124.18B1: 88.86B2: 75.86B3: 59.94
논문 코드

데이터셋

42개
VizWiz 2020 test-devCOCO Captionsnocaps in-domainnocaps out-of-domainnocaps entirenocaps near-domainCOCO (Common Objects in Context)VizWiz 2020 testScanRefer Datasetnocaps-XD entireTextCaps 2020nocaps-XD in-domain

논문

라이브러리

Papers with Code 리더보드 row에 연결된 구현 저장소입니다.

github.com/microsoft/GenerativeImage2Text 1 논문
github.com/FerryHuang/SimVLM 1 논문
github.com/google-research/big_vision 1 논문
github.com/yulong-XJTU/SimVLM 1 논문

하위 태스크

3D 3D dense captioning 2 데이터셋 · 22 rows SS Semi Supervised Learning for Image Captioning 3 데이터셋 · 3 rows HI Hindi Image Captioning 2 데이터셋 · 2 rows RC Relational Captioning 1 데이터셋 · 2 rows AI Aesthetic Image Captioning 0 데이터셋 · 0 rows PF Patent Figure Description Generation 0 데이터셋 · 0 rows VI Vietnamese Image Captioning 0 데이터셋 · 0 rows CI controllable image captioning 0 데이터셋 · 0 rows