Inverse Reinforcement Learning with Sub-optimal Experts
원문 초록
Inverse Reinforcement Learning (IRL) techniques deal with the problem of deducing a reward function that explains the behavior of an expert agent who is assumed to act optimally in an underlying unknown task. In several problems of interest, however, it is possible to observe the behavior of multiple experts with different degree of optimality (e.g., racing drivers whose skills ranges from amateurs to professionals). For this reason, in this work, we extend the IRL formulation to problems where, in addition to demonstrations from the optimal agent, we can observe the behavior of multiple sub-optimal experts. Given this problem, we first study the theoretical properties of the class of reward functions that are compatible with a given set of experts, i.e., the feasible reward set. Our results show that the presence of multiple sub-optimal experts can significantly shrink the set of compatible rewards. Furthermore, we study the statistical complexity of estimating the feasible reward set with a generative model. To this end, we analyze a uniform sampling algorithm that results in being minimax optimal whenever the sub-optimal experts' performance level is sufficiently close to the one of the optimal agent.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Reasoning on a Budget: Miniaturizing DeepSeek R1 with SFT-GRPO Alignment for Instruction-Tuned LLMs
Meta Reinforcement Learning for Multi-Task Offloading in Vehicular Edge Computing
Investigating the Performance and Reliability, of the Q-Learning Algorithm in Various Unknown Environments
Margin Trader: A Reinforcement Learning Framework for Portfolio Management with Margin and Constraints
Catch: Collaborative Feature Set Search for Automated Feature Engineering
Solving Dynamic Traveling Salesman Problems With Deep Reinforcement Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.