Learning to Generate All Feasible Actions
원문 초록
Modern cyber-physical systems are becoming increasingly complex to model, thus motivating data-driven techniques such as reinforcement learning (RL) to find appropriate control agents. However, most systems are subject to hard constraints such as safety or operational bounds. Typically, to learn to satisfy these constraints, the agent must violate them systematically, which is computationally prohibitive in most systems. Recent efforts aim to utilize feasibility models that assess whether a proposed action is feasible to avoid applying the agent's infeasible action proposals to the system. However, these efforts focus on guaranteeing constraint satisfaction rather than the agent's learning efficiency. To improve the learning process, we introduce action mapping, a novel approach that divides the learning process into two steps: first learn feasibility and subsequently, the objective by mapping actions into the sets of feasible actions. This paper focuses on the feasibility part by learning to generate all feasible actions through self-supervised querying of the feasibility model. We train the agent by formulating the problem as a distribution matching problem and deriving gradient estimators for different divergences. Through an illustrative example, a robotic path planning scenario, and a robotic grasping simulation, we demonstrate the agent's proficiency in generating actions across disconnected feasible action sets. By addressing the feasibility step, this paper makes it possible to focus future work on the objective part of action mapping, paving the way for an RL framework that is both safe and efficient.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
MasHost Builds It All: Autonomous Multi-Agent System Directed by Reinforcement Learning
High-Throughput Distributed Reinforcement Learning via Adaptive Policy Synchronization
Personalized Exercise Recommendation with Semantically-Grounded Knowledge Tracing
Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities
Exploring the robustness of TractOracle methods in RL-based tractography
Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination
토론
이 논문과 연결된 포럼 글이 아직 없습니다.