PK PyTorchKR Papers
Papers with Code · CC BY-SA 4.0

Question Answering

2,670 평가 기록 261 datasets

Question answering can be segmented into domain-specific tasks like community question answering and knowledge-base question answering. Popular benchmark datasets for evaluation question answering systems include SQuAD, HotPotQA, bAbI, TriviaQA, WikiQA, and many others. Models for question answering are typically evaluated on metrics like EM and F1. Some recent top performing models are T5 and XLNet.

( Image credit: SQuAD )

개요는 각 항목의 일부입니다. 위 탭에서 전체 기록을 페이지별로 탐색할 수 있습니다.

벤치마크

Papers with Code 아카이브의 데이터셋별 기록 예시입니다. 평가 조건과 지표 방향이 검증되지 않아 순위를 의미하지 않습니다.

Dataset기록된 모델 예시MetricPaperCode
IE-Net (ensemble)
EM: 90.939F1: 93.214
- -
{ANNA} (single model)
EM: 90.622F1: 95.719
- -
Gemini 2.0 Flash Experimental
Accuracy: 89.7
- -
Beam Retrieval
ANS-EM: 0.727ANS-F1: 0.850SUP-EM: 0.663SUP-F1: 0.901
논문 코드
Unicorn 11B (fine-tuned)
Accuracy: 90.1
논문 코드
Mistral-Nemo 12B (HPT)
Accuracy: 99.87
논문 코드
PaLM 540B (finetuned)
Accuracy: 100
논문 코드
Claude 2 (few-shot, k=5)
EM: 87.5
논문 -
T5-11B
EM: 90.06F1: 95.64
논문 코드
Atlas (full, Wiki-dec-2018 index)
EM: 64.0
논문 코드
GPT-4 + knowledge base
Accuracy: 95.9
- -
CoA
EM: 70.7
논문 코드

데이터셋

261개
SQuAD2.0SQuAD1.1MATHHotpotQAPIQABoolQCOPATriviaQASQuAD1.1 devNatural QuestionsOpenBookQAWebQuestions

논문

라이브러리

Papers with Code 리더보드 row에 연결된 구현 저장소입니다.

github.com/ofirpress/self-ask 1 논문
github.com/oriyor/reasoning-on-cots 1 논문
github.com/oriyor/ret-robust 1 논문
github.com/ruc-nlpir/flashrag 1 논문

하위 태스크

MQ Mathematical Question Answering 15 데이터셋 · 255 rows ZV Zero-Shot Video Question Answer 17 데이터셋 · 239 rows MC Multiple Choice Question Answering (MCQA) 31 데이터셋 · 179 rows OQ Open-Domain Question Answering 15 데이터셋 · 109 rows KB Knowledge Base Question Answering 10 데이터셋 · 44 rows CQ Cross-Lingual Question Answering 3 데이터셋 · 18 rows AS Answer Selection 6 데이터셋 · 10 rows SQ Science Question Answering 1 데이터셋 · 10 rows TQ Temporal/Casual QA 1 데이터셋 · 8 rows GQ Generative Question Answering 2 데이터셋 · 7 rows LR Logical Reasoning Question Answering 1 데이터셋 · 3 rows CQ Community Question Answering 2 데이터셋 · 2 rows