Time-Scale Separation in Q-Learning: Extending TD($\triangle$) for Action-Value Function Decomposition
원문 초록
Q-Learning is a fundamental off-policy reinforcement learning (RL) algorithm that has the objective of approximating action-value functions in order to learn optimal policies. Nonetheless, it has difficulties in reconciling bias with variance, particularly in the context of long-term rewards. This paper introduces Q($\Delta$)-Learning, an extension of TD($\Delta$) for the Q-Learning framework. TD($\Delta$) facilitates efficient learning over several time scales by breaking the Q($\Delta$)-function into distinct discount factors. This approach offers improved learning stability and scalability, especially for long-term tasks where discounting bias may impede convergence. Our methodology guarantees that each element of the Q($\Delta$)-function is acquired individually, facilitating expedited convergence on shorter time scales and enhancing the learning of extended time scales. We demonstrate through theoretical analysis and practical evaluations on standard benchmarks like Atari that Q($\Delta$)-Learning surpasses conventional Q-Learning and TD learning methods in both tabular and deep RL environments.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Personalized Exercise Recommendation with Semantically-Grounded Knowledge Tracing
Reinforcement Learning-Based Policy Optimisation For Heterogeneous Radio Access
ReinDSplit: Reinforced Dynamic Split Learning for Pest Recognition in Precision Agriculture
Regret-Optimal Q-Learning with Low Cost for Single-Agent and Federated Reinforcement Learning
Reinforcement Learning for Hanabi
PGDQN: Preference-Guided Deep Q-Network
토론
이 논문과 연결된 포럼 글이 아직 없습니다.