On the Convergence Rates of Policy Gradient Methods
원문 초록
We consider infinite-horizon discounted Markov decision problems with finite state and action spaces and study the convergence rates of the projected policy gradient method and a general class of policy mirror descent methods, all with direct parametrization in the policy space. First, we develop a theory of weak gradient-mapping dominance and use it to prove sharper sublinear convergence rate of the projected policy gradient method. Then we show that with geometrically increasing step sizes, a general class of policy mirror descent methods, including the natural policy gradient method and a projected Q-descent method, all enjoy a linear rate of convergence without relying on entropy or other strongly convex regularization. Finally, we also analyze the convergence rate of an inexact policy mirror descent method and estimate its sample complexity under a simple generative model.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Models
Hierarchical Policy-Gradient Reinforcement Learning for Multi-Agent Shepherding Control of Non-Cohesive Targets
Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
Reevaluating Policy Gradient Methods for Imperfect-Information Games
Fine-Tuning Discrete Diffusion Models with Policy Gradient Methods
Divergence-Augmented Policy Optimization
토론
이 논문과 연결된 포럼 글이 아직 없습니다.