arXiv:2007.11120
On Linear Convergence of Policy Gradient Methods for Finite MDPs
원문 초록
We revisit the finite time analysis of policy gradient methods in the one of the simplest settings: finite state and action MDPs with a policy class consisting of all stochastic policies and with exact gradient evaluations. There has been some recent work viewing this setting as an instance of smooth non-linear optimization problems and showing sub-linear convergence rates with small step-sizes. Here, we take a different perspective based on connections with policy iteration and show that many variants of policy gradient methods succeed with large step-sizes and attain a linear rate of convergence.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Models
Hierarchical Policy-Gradient Reinforcement Learning for Multi-Agent Shepherding Control of Non-Cohesive Targets
Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
Reevaluating Policy Gradient Methods for Imperfect-Information Games
Fine-Tuning Discrete Diffusion Models with Policy Gradient Methods
Divergence-Augmented Policy Optimization
토론
이 논문과 연결된 포럼 글이 아직 없습니다.