Motion Reasoning for Goal-Based Imitation Learning
원문 초록
We address goal-based imitation learning, where the aim is to output the symbolic goal from a third-person video demonstration. This enables the robot to plan for execution and reproduce the same goal in a completely different environment. The key challenge is that the goal of a video demonstration is often ambiguous at the level of semantic actions. The human demonstrators might unintentionally achieve certain subgoals in the demonstrations with their actions. Our main contribution is to propose a motion reasoning framework that combines task and motion planning to disambiguate the true intention of the demonstrator in the video demonstration. This allows us to robustly recognize the goals that cannot be disambiguated by previous action-based approaches. We evaluate our approach by collecting a dataset of 96 video demonstrations in a mockup kitchen environment. We show that our motion reasoning plays an important role in recognizing the actual goal of the demonstrator and improves the success rate by over 20%. We further show that by using the automatically inferred goal from the video demonstration, our robot is able to reproduce the same task in a real kitchen environment.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Prime the search: Using large language models for guiding geometric task and motion planning by warm-starting tree search
Ark: An Open-source Python-based Framework for Robot Learning
Language-Grounded Hierarchical Planning and Execution with Multi-Robot 3D Scene Graphs
Understanding Physical Properties of Unseen Deformable Objects by Leveraging Large Language Models and Robot Actions
Grounded Vision-Language Interpreter for Integrated Task and Motion Planning
From Motion to Behavior: Hierarchical Modeling of Humanoid Generative Behavior Control
토론
이 논문과 연결된 포럼 글이 아직 없습니다.