Papers with Code · CC BY-SA 4.0

MBPP

아카이브 평가 99건 · 마지막 적재/변경 2026-10-30

평가 기록

과거 수집 기록이며 최신 순위가 아닙니다. 데이터셋 버전·split·실험 조건과 지표 방향이 검증되지 않은 결과는 직접 비교할 수 없습니다. 날짜는 논문 날짜이며 최신 평가일을 뜻하지 않습니다.

표가 화면보다 넓으면 좌우로 스크롤하여 논문과 구현 링크를 확인하세요.

원본 기록 순서 · 순위 미부여
모델	원본 지표·값	논문	구현
DeepSeek-Coder-Instruct 33B (few-shot)	Accuracy 70	DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence 2024-01-25	코드 1
GPT-3.5 Turbo + INTERVENOR	Accuracy 69.8	INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repair 2023-11-16	코드 1
code-davinci-002 175B + LEVER	Accuracy 68.9	LEVER: Learning to Verify Language-to-Code Generation with Execution 2023-02-16	코드 1
code-davinci-002 175B + CodeT	Accuracy 67.7	CodeT: Code Generation with Generated Tests 2022-07-21	코드 1
GPT-3.5 Turbo (3-shot)	Accuracy 67.6	Teaching Large Language Models to Self-Debug 2023-04-11	코드 1 · 코드 2
code-davinci-002 175B + Reviewer	Accuracy 66.9	Coder Reviewer Reranking for Code Generation 2022-11-29	코드 1
code-davinci-002 175B + Coder-Reviewer	Accuracy 66.4	Coder Reviewer Reranking for Code Generation 2022-11-29	코드 1
StarCoder2-15B	Accuracy 66.2	StarCoder 2 and The Stack v2: The Next Generation 2024-02-29	코드 1 · 코드 2 · 코드 3 · 코드 4
DeepSeek-Coder-Base 33B (few-shot)	Accuracy 66	DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence 2024-01-25	코드 1
Code Llama - Python 70B (3-shot)	Accuracy 65.5	Code Llama: Open Foundation Models for Code 2023-08-24	코드 1 · 코드 2
DeepSeek-Coder-Instruct 6.7B (few-shot)	Accuracy 65.4	DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence 2024-01-25	코드 1
code-davinci-002 175B + MBR-Exec	Accuracy 63	Coder Reviewer Reranking for Code Generation 2022-11-29	코드 1
Code Llama 70B (3-shot)	Accuracy 62.4	Code Llama: Open Foundation Models for Code 2023-08-24	코드 1 · 코드 2
Code Llama - Instruct 70B (3-shot)	Accuracy 62.2	Code Llama: Open Foundation Models for Code 2023-08-24	코드 1 · 코드 2
code-davinci-001 175B + CodeT	Accuracy 61.9	CodeT: Code Generation with Generated Tests 2022-07-21	코드 1
code-davinci-002 175B (3-shot)	Accuracy 61.4	Teaching Large Language Models to Self-Debug 2023-04-11	코드 1 · 코드 2
Unnatural Code Llama 34B (3-shot)	Accuracy 61.2	Code Llama: Open Foundation Models for Code 2023-08-24	코드 1 · 코드 2
Mixtral 8x7B (3-shot)	Accuracy 60.7	Mixtral of Experts 2024-01-08	코드 1 · 코드 2 · 코드 3 · 코드 4 · 코드 5 · 코드 6
DeepSeek-Coder-Base 6.7B (few-shot)	Accuracy 60.6	DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence 2024-01-25	코드 1
code-davinci-001 175B + MBR-Exec	Accuracy 58.2	Natural Language to Code Translation with Execution 2022-04-25	코드 1
Code Llama - Instruct 34B (3-shot)	Accuracy 57	Code Llama: Open Foundation Models for Code 2023-08-24	코드 1 · 코드 2
Code Llama - Python 34B (3-shot)	Accuracy 56.2	Code Llama: Open Foundation Models for Code 2023-08-24	코드 1 · 코드 2
code-cushman-001 12B (CodeT)	Accuracy 55.4	CodeT: Code Generation with Generated Tests 2022-07-21	코드 1
Code Llama 34B (3-shot)	Accuracy 55	Code Llama: Open Foundation Models for Code 2023-08-24	코드 1 · 코드 2
StarCoder 15.5B (Self-Debugging with unit tests + trace)	Accuracy 53.2	Teaching Large Language Models to Self-Debug 2023-04-11	코드 1 · 코드 2
StarCoder 15.5B	Accuracy 52.7	StarCoder: may the source be with you! 2023-05-09	코드 1 · 코드 2 · 코드 3 · 코드 4
GPT-3.5 Turbo	Accuracy 52.2	Code Llama: Open Foundation Models for Code 2023-08-24	코드 1 · 코드 2
WizardCoder 15B	Accuracy 51.8	WizardCoder: Empowering Code Large Language Models with Evol-Instruct 2023-06-14	코드 1 · 코드 2 · 코드 3 · 코드 4
PaLM 2-S* (few-shot)	Accuracy 50	PaLM 2 Technical Report 2023-05-17	코드 1
CodeGen-Mono 16B + CodeT	Accuracy 49.5	CodeT: Code Generation with Generated Tests 2022-07-21	코드 1