Gravity-Bench-v1: A Benchmark on Gravitational Physics Discovery for Agents
원문 초록
Modern science emerged from reasoning over repeatedly-observed planetary motions. We present Gravity-Bench-v1, an environment-based benchmark that challenges AI agents on tasks that parallel this historical development. Gravity-Bench-v1 evaluates agents on the discovery of physics concealed within a dynamic environment, using rigorous gravitational dynamics simulations. Gravity-Bench includes out-of-distribution cases, i.e. with physics that deviates from the real world, to evaluate true scientific generalization capabilities. Agents must plan to collect data within an experimental budget and must perform a dynamic form of data analysis and reasoning to solve tasks efficiently. Our benchmark admits an open-ended space of solutions. Reference solutions for each task are provided to calibrate AI performance against human expertise. Technically at an upper-undergraduate level, our benchmark proves challenging to baseline AI agents. Gravity-Bench-v1 and planned extensions should help map out AI progress towards scientific discovery capabilities.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Open Source Planning & Control System with Language Agents for Autonomous Scientific Discovery
STRUCTSENSE: A Task-Agnostic Agentic Framework for Structured Information Extraction with Human-In-The-Loop Evaluation and Benchmarking
From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking
LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Research
ClimateChat: Designing Data and Methods for Instruction Tuning LLMs to Answer Climate Change Queries
HSG-12M: A Large-Scale Spatial Multigraph Dataset
토론
이 논문과 연결된 포럼 글이 아직 없습니다.