An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents
원문 초록
Reinforcement learning (RL) has demonstrated strong potential in training large language models (LLMs) capable of complex reasoning for real-world problem solving. More recently, RL has been leveraged to create sophisticated LLM-based search agents that adeptly combine reasoning with search engine use. While the use of RL for training search agents is promising, the optimal design of such agents remains not fully understood. In particular, key factors -- such as (1) reward formulation, (2) the choice and characteristics of the underlying LLM, and (3) the role of the search engine in the RL process -- require further investigation. In this work, we conduct comprehensive empirical studies to systematically investigate these and offer actionable insights. We highlight several key findings: format rewards are effective in improving final performance, whereas intermediate retrieval rewards have limited impact; the scale and initialization of the LLM (general-purpose vs. reasoning-specialized) significantly influence RL outcomes; and the choice of search engine plays a critical role in shaping RL training dynamics and the robustness of the trained agent during inference. These establish important guidelines for successfully building and deploying LLM-based search agents in real-world applications. Code is available at https://github.com/PeterGriffinJin/Search-R1.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Refining Few-Step Text-to-Multiview Diffusion via Reinforcement Learning
Fully Offline Reinforcement Learning
High-Throughput Distributed Reinforcement Learning via Adaptive Policy Synchronization
Personalized Exercise Recommendation with Semantically-Grounded Knowledge Tracing
Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities
Exploring the robustness of TractOracle methods in RL-based tractography
토론
이 논문과 연결된 포럼 글이 아직 없습니다.