Is Risk-Sensitive Reinforcement Learning Properly Resolved?
원문 초록
Due to the nature of risk management in learning applicable policies, risk-sensitive reinforcement learning (RSRL) has been realized as an important direction. RSRL is usually achieved by learning risk-sensitive objectives characterized by various risk measures, under the framework of distributional reinforcement learning. However, it remains unclear if the distributional Bellman operator properly optimizes the RSRL objective in the sense of risk measures. In this paper, we prove that the existing RSRL methods do not achieve unbiased optimization and can not guarantee optimality or even improvements regarding risk measures over accumulated return distributions. To remedy this issue, we further propose a novel algorithm, namely Trajectory Q-Learning (TQL), for RSRL problems with provable convergence to the optimal policy. Based on our new learning architecture, we are free to introduce a general and practical implementation for different risk measures to learn disparate risk-sensitive policies. In the experiments, we verify the learnability of our algorithm and show how our method effectively achieves better performances toward risk-sensitive objectives.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Autonomous Resource Management in Microservice Systems via Reinforcement Learning
Distributional Reinforcement Learning on Path-dependent Options
CTRLS: Chain-of-Thought Reasoning via Latent State-Transition
Explainable AI for Radar Resource Management: Modified LIME in Deep Reinforcement Learning
Multi-Objective Reinforcement Learning for Cognitive Radar Resource Management
Multi-Agent Reinforcement Learning for Autonomous Multi-Satellite Earth Observation: A Realistic Case Study
토론
이 논문과 연결된 포럼 글이 아직 없습니다.