Policy Gradient Methods for Off-policy Control
원문 초록
Off-policy learning refers to the problem of learning the value function of a way of behaving, or policy, while following a different policy. Gradient-based off-policy learning algorithms, such as GTD and TDC/GQ, converge even when using function approximation and incremental updates. However, they have been developed for the case of a fixed behavior policy. In control problems, one would like to adapt the behavior policy over time to become more greedy with respect to the existing value function. In this paper, we present the first gradient-based learning algorithms for this problem, which rely on the framework of policy gradient in order to modify the behavior policy. We present derivations of the algorithms, a convergence theorem, and empirical evidence showing that they compare favorably to existing approaches.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Models
Hierarchical Policy-Gradient Reinforcement Learning for Multi-Agent Shepherding Control of Non-Cohesive Targets
Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
Reevaluating Policy Gradient Methods for Imperfect-Information Games
Fine-Tuning Discrete Diffusion Models with Policy Gradient Methods
Divergence-Augmented Policy Optimization
토론
이 논문과 연결된 포럼 글이 아직 없습니다.