Papers with Code · CC BY-SA 4.0

Reading Comprehension

142 평가 기록 28 datasets

Most current question answering datasets frame the task as reading comprehension where the question is about a paragraph or document and the answer often is a span in the document.

Some specific tasks of reading comprehension include multi-modal machine reading comprehension and textual machine reading comprehension, among others. In the literature, machine reading comprehension can be divide into four categories: cloze style, multiple choice, span prediction, and free-form answer. Read more about each category here.

Benchmark datasets used for testing a model's reading comprehension abilities include MovieQA, ReCoRD, and RACE, among others.

The Machine Reading group at UCL also provides an overview of reading comprehension tasks.

Figure source: A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets

개요는 각 항목의 일부입니다. 위 탭에서 전체 기록을 페이지별로 탐색할 수 있습니다.

벤치마크

Papers with Code 아카이브의 데이터셋별 기록 예시입니다. 평가 조건과 지표 방향이 검증되지 않아 순위를 의미하지 않습니다.

Dataset기록된 모델 예시MetricPaperCode
Rational Reasoner / IDOL
Test: 80.6
논문 코드
ALBERT (Ensemble)
Accuracy: 91.4
논문 -
Golden Transformer
Average F1: 0.941EM : 0.819
- -
Human
Accuracy (20 classes): 85.51Accuracy (Binary): 94.72
논문 코드
PaCE
F1: 63.8Precision: 63.3Recall: 68
논문 코드
PaCE
F1: 77.6
논문 코드
RoBERTa-Large
D(BiDAF): F1: 74.1D(BERT): F1: 65.5D(RoBERTa): F1: 53.4Overall: F1: 64.4
논문 코드
ASA + RoBERTa
Accuracy: 69.2
논문 코드
XLNet-large
Accuracy: 56.0Accuracy (easy): 75.7Accuracy (hard): 40.5
논문 코드
Chinchilla-70B (few-shot, k=5)
Accuracy : 92.8
논문 코드
Chinchilla-70B (few-shot, k=5)
Accuracy : 49.4
논문 코드
Chinchilla-70B (few-shot, k=5)
Accuracy: 52.6
논문 코드

데이터셋

28개

논문

라이브러리

Papers with Code 리더보드 row에 연결된 구현 저장소입니다.

하위 태스크