Algorithms for Batch Hierarchical Reinforcement Learning
원문 초록
Hierarchical Reinforcement Learning (HRL) exploits temporal abstraction to solve large Markov Decision Processes (MDP) and provide transferable subtask policies. In this paper, we introduce an off-policy HRL algorithm: Hierarchical Q-value Iteration (HQI). We show that it is possible to effectively learn recursive optimal policies for any valid hierarchical decomposition of the original MDP, given a fixed dataset collected from a flat stochastic behavioral policy. We first formally prove the convergence of the algorithm for tabular MDP. Then our experiments on the Taxi domain show that HQI converges faster than a flat Q-value Iteration and enjoys easy state abstraction. Also, we demonstrate that our algorithm is able to learn optimal policies for different hierarchical structures from the same fixed dataset, which enables model comparison without recollecting data.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Hierarchical Reinforcement Learning and Value Optimization for Challenging Quadruped Locomotion
HiLight: A Hierarchical Reinforcement Learning Framework with Global Adversarial Guidance for Large-Scale Traffic Signal Control
Policy-Based Trajectory Clustering in Offline Reinforcement Learning
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
High-Throughput Distributed Reinforcement Learning via Adaptive Policy Synchronization
Exploring the robustness of TractOracle methods in RL-based tractography
토론
이 논문과 연결된 포럼 글이 아직 없습니다.