SuperCLUE-Math6: Graded Multi-Step Math Reasoning Benchmark for LLMs in Chinese
원문 초록
We introduce SuperCLUE-Math6(SC-Math6), a new benchmark dataset to evaluate the mathematical reasoning abilities of Chinese language models. SC-Math6 is designed as an upgraded Chinese version of the GSM8K dataset with enhanced difficulty, diversity, and application scope. It consists of over 2000 mathematical word problems requiring multi-step reasoning and providing natural language solutions. We propose an innovative scheme to quantify the reasoning capability of large models based on performance over problems with different reasoning steps. Experiments on 13 representative Chinese models demonstrate a clear stratification of reasoning levels, with top models like GPT-4 showing superior performance. SC-Math6 fills the gap in Chinese mathematical reasoning benchmarks and provides a comprehensive testbed to advance the intelligence of Chinese language models.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
CoRE: Enhancing Metacognition with Label-free Self-evaluation in LRMs
Xolver: Multi-Agent Reasoning with Holistic Experience Learning Just Like an Olympiad Team
Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination
A Practical Two-Stage Recipe for Mathematical LLMs: Maximizing Accuracy with SFT and Efficiency with Reinforcement Learning
Confucius3-Math: A Lightweight High-Performance Reasoning LLM for Chinese K-12 Mathematics Learning
AgentGroupChat-V2: Divide-and-Conquer Is What LLM-Based Multi-Agent System Need
토론
이 논문과 연결된 포럼 글이 아직 없습니다.