Reinforcement Learning based Constrained Optimal Control: an Interpretable Reward Design
원문 초록
This paper presents an interpretable reward design framework for reinforcement learning based constrained optimal control problems with state and terminal constraints. The problem is formalized within a standard partially observable Markov decision process framework. The reward function is constructed from four weighted components: a terminal constraint reward, a guidance reward, a penalty for state constraint violations, and a cost reduction incentive reward. A theoretically justified reward design is then presented, which establishes bounds on the weights of the components. This approach ensures that constraints are satisfied and objectives are optimized while mitigating numerical instability. Acknowledging the importance of prior knowledge in reward design, we sequentially solve two subproblems, using each solution to inform the reward design for the subsequent problem. Subsequently, we integrate reinforcement learning with curriculum learning, utilizing policies derived from simpler subproblems to assist in tackling more complex challenges, thereby facilitating convergence. The framework is evaluated against original and randomly weighted reward designs in a multi-agent particle environment. Experimental results demonstrate that the proposed approach significantly enhances satisfaction of terminal and state constraints and optimization of control cost.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Reasoning on a Budget: Miniaturizing DeepSeek R1 with SFT-GRPO Alignment for Instruction-Tuned LLMs
Meta Reinforcement Learning for Multi-Task Offloading in Vehicular Edge Computing
Investigating the Performance and Reliability, of the Q-Learning Algorithm in Various Unknown Environments
Margin Trader: A Reinforcement Learning Framework for Portfolio Management with Margin and Constraints
Catch: Collaborative Feature Set Search for Automated Feature Engineering
Solving Dynamic Traveling Salesman Problems With Deep Reinforcement Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.