Assured RL: Reinforcement Learning with Almost Sure Constraints
원문 초록
We consider the problem of finding optimal policies for a Markov Decision Process with almost sure constraints on state transitions and action triplets. We define value and action-value functions that satisfy a barrier-based decomposition which allows for the identification of feasible policies independently of the reward process. We prove that, given a policy {\pi}, certifying whether certain state-action pairs lead to feasible trajectories under {\pi} is equivalent to solving an auxiliary problem aimed at finding the probability of performing an unfeasible transition. Using this interpretation,we develop a Barrier-learning algorithm, based on Q-Learning, that identifies such unsafe state-action pairs. Our analysis motivates the need to enhance the Reinforcement Learning (RL) framework with an additional signal, besides rewards, called here damage function that provides feasibility information and enables the solution of RL problems with model-free constraints. Moreover, our Barrier-learning algorithm wraps around existing RL algorithms, such as Q-Learning and SARSA, giving them the ability to solve almost-surely constrained problems.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Task Independent Capsule-Based Agents for Deep Q-Learning
Variation-resistant Q-learning: Controlling and Utilizing Estimation Bias in Reinforcement Learning for Better Performance
Robust Multi-Agent Reinforcement Learning with Model Uncertainty
Deep Reinforcement Learning Based Power Allocation for D2D Network
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
Investigating the Performance and Reliability, of the Q-Learning Algorithm in Various Unknown Environments
토론
이 논문과 연결된 포럼 글이 아직 없습니다.