Provably Efficient Exploration in Policy Optimization
원문 초록
While policy-based reinforcement learning (RL) achieves tremendous successes in practice, it is significantly less understood in theory, especially compared with value-based RL. In particular, it remains elusive how to design a provably efficient policy optimization algorithm that incorporates exploration. To bridge such a gap, this paper proposes an Optimistic variant of the Proximal Policy Optimization algorithm (OPPO), which follows an ``optimistic version'' of the policy gradient direction. This paper proves that, in the problem of episodic Markov decision process with linear function approximation, unknown transition, and adversarial reward with full-information feedback, OPPO achieves $\tilde{O}(\sqrt{d^2 H^3 T} )$ regret. Here $d$ is the feature dimension, $H$ is the episode horizon, and $T$ is the total number of steps. To the best of our knowledge, OPPO is the first provably efficient policy optimization algorithm that explores.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Multirobot Coverage of Modular Environments
Margin Trader: A Reinforcement Learning Framework for Portfolio Management with Margin and Constraints
PGDQN: Preference-Guided Deep Q-Network
Task Independent Capsule-Based Agents for Deep Q-Learning
Accelerating lifelong reinforcement learning via reshaping rewards
Federated Model Search via Reinforcement Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.