Is Q-learning an Ill-posed Problem?
원문 초록
This paper investigates the instability of Q-learning in continuous environments, a challenge frequently encountered by practitioners. Traditionally, this instability is attributed to bootstrapping and regression model errors. Using a representative reinforcement learning benchmark, we systematically examine the effects of bootstrapping and model inaccuracies by incrementally eliminating these potential error sources. Our findings reveal that even in relatively simple benchmarks, the fundamental task of Q-learning - iteratively learning a Q-function from policy-specific target values - can be inherently ill-posed and prone to failure. These insights cast doubt on the reliability of Q-learning as a universal solution for reinforcement learning problems.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Reinforcement Learning-Based Policy Optimisation For Heterogeneous Radio Access
Reinforcement Learning for Hanabi
Reinforcement Learning for Stock Transactions
Offline Guarded Safe Reinforcement Learning for Medical Treatment Optimization Strategies
Investigating the Performance and Reliability, of the Q-Learning Algorithm in Various Unknown Environments
Task Independent Capsule-Based Agents for Deep Q-Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.