← Logical Reasoning

Papers with Code · CC BY-SA 4.0

LingOly

아카이브 평가 11건 · 마지막 적재/변경 2026-10-30

원본 데이터셋 링크 1

평가 기록

과거 수집 기록이며 최신 순위가 아닙니다. 데이터셋 버전·split·실험 조건과 지표 방향이 검증되지 않은 결과는 직접 비교할 수 없습니다. 날짜는 논문 날짜이며 최신 평가일을 뜻하지 않습니다.

표가 화면보다 넓으면 좌우로 스크롤하여 논문과 구현 링크를 확인하세요.

원본 기록 순서 · 순위 미부여
모델	원본 지표·값	논문	구현
Claude Opus	Exact Match Accuracy 46.3% Delta_NoContext 28.8%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages 2024-06-10	코드 1
GPT-4o	Exact Match Accuracy 37.6% Delta_NoContext 25.1%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages 2024-06-10	코드 1
Gemini 1.5 Pro	Exact Match Accuracy 32.1% Delta_NoContext 23.4%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages 2024-06-10	코드 1
GPT-4	Exact Match Accuracy 33.4% Delta_NoContext 21.5%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages 2024-06-10	코드 1
Command R+	Exact Match Accuracy 21.5% Delta_NoContext 11.6%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages 2024-06-10	코드 1
GPT-3.5	Exact Match Accuracy 21.2% Delta_NoContext 11.2%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages 2024-06-10	코드 1
Mixtral 8x7B	Exact Match Accuracy 14.2% Delta_NoContext 6.4%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages 2024-06-10	코드 1
Llama 3 8B	Exact Match Accuracy 11.4% Delta_NoContext 4.9%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages 2024-06-10	코드 1
Llama 3 70B	Exact Match Accuracy 10.3% Delta_NoContext 2.9%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages 2024-06-10	코드 1
Gemma 7B	Exact Match Accuracy 4.9% Delta_NoContext 2.2%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages 2024-06-10	코드 1
Llama 2 70B	Exact Match Accuracy 6.4% Delta_NoContext 1.1%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages 2024-06-10	코드 1