Proximal Policy Gradient: PPO with Policy Gradient
원문 초록
In this paper, we propose a new algorithm PPG (Proximal Policy Gradient), which is close to both VPG (vanilla policy gradient) and PPO (proximal policy optimization). The PPG objective is a partial variation of the VPG objective and the gradient of the PPG objective is exactly same as the gradient of the VPG objective. To increase the number of policy update iterations, we introduce the advantage-policy plane and design a new clipping strategy. We perform experiments in OpenAI Gym and Bullet robotics environments for ten random seeds. The performance of PPG is comparable to PPO, and the entropy decays slower than PPG. Thus we show that performance similar to PPO can be obtained by using the gradient formula from the original policy gradient theorem.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
HDDLGym: A Tool for Studying Multi-Agent Hierarchical Problems Defined in HDDL with OpenAI Gym
Improving the Data-efficiency of Reinforcement Learning by Warm-starting with LLM
Mining-Gym: A Configurable RL Benchmarking Environment for Truck Dispatch Scheduling
A quantum-classical reinforcement learning model to play Atari games
Creating Hierarchical Dispositions of Needs in an Agent
Double Successive Over-Relaxation Q-Learning with an Extension to Deep Reinforcement Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.