Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise
원문 초록
This paper establishes the first almost sure convergence rate and the first maximal concentration bound with exponential tails for general contractive stochastic approximation algorithms with Markovian noise. As a corollary, we also obtain convergence rates in $L^p$. Key to our successes is a novel discretization of the mean ODE of stochastic approximation algorithms using intervals with diminishing (instead of constant) length. As applications, we provide the first almost sure convergence rate for $Q$-learning with Markovian samples without count-based learning rates. We also provide the first concentration bound for off-policy temporal difference learning with Markovian samples.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Q-Learning under Finite Model Uncertainty
Personalized Exercise Recommendation with Semantically-Grounded Knowledge Tracing
ADDQ: Adaptive Distributional Double Q-Learning
Inverse Q-Learning Done Right: Offline Imitation Learning in $Q^π$-Realizable MDPs
Distributionally Robust Deep Q-Learning
A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows
토론
이 논문과 연결된 포럼 글이 아직 없습니다.