Target Network and Truncation Overcome The Deadly Triad in $Q$-Learning
원문 초록
$Q$-learning with function approximation is one of the most empirically successful while theoretically mysterious reinforcement learning (RL) algorithms, and was identified in Sutton (1999) as one of the most important theoretical open problems in the RL community. Even in the basic linear function approximation setting, there are well-known divergent examples. In this work, we show that \textit{target network} and \textit{truncation} together are enough to provably stabilize $Q$-learning with linear function approximation, and we establish the finite-sample guarantees. The result implies an $O(\epsilon^{-2})$ sample complexity up to a function approximation error. Moreover, our results do not require strong assumptions or modifying the problem parameters as in existing literature.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Task Independent Capsule-Based Agents for Deep Q-Learning
Variation-resistant Q-learning: Controlling and Utilizing Estimation Bias in Reinforcement Learning for Better Performance
Robust Multi-Agent Reinforcement Learning with Model Uncertainty
Deep Reinforcement Learning Based Power Allocation for D2D Network
Dependency-Aware Computation Offloading in Mobile Edge Computing: A Reinforcement Learning Approach
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
토론
이 논문과 연결된 포럼 글이 아직 없습니다.