Differential Temporal Difference Learning
원문 초록
Value functions derived from Markov decision processes arise as a central component of algorithms as well as performance metrics in many statistics and engineering applications of machine learning techniques. Computation of the solution to the associated Bellman equations is challenging in most practical cases of interest. A popular class of approximation techniques, known as Temporal Difference (TD) learning algorithms, are an important sub-class of general reinforcement learning methods. The algorithms introduced in this paper are intended to resolve two well-known difficulties of TD-learning approaches: Their slow convergence due to very high variance, and the fact that, for the problem of computing the relative value function, consistent algorithms exist only in special cases. First we show that the gradients of these value functions admit a representation that lends itself to algorithm design. Based on this result, a new class of differential TD-learning algorithms is introduced. For Markovian models on Euclidean space with smooth dynamics, the algorithms are shown to be consistent under general conditions. Numerical results show dramatic variance reduction when compared to standard methods.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learning
PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning
D3PG: Dirichlet DDPG for Task Partitioning and Offloading With Constrained Hybrid Action Space in Mobile-Edge Computing
A State Representation Dueling Network for Deep Reinforcement Learning
Refining Few-Step Text-to-Multiview Diffusion via Reinforcement Learning
Fully Offline Reinforcement Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.