Phone2Proc: Bringing Robust Robots Into Our Chaotic World
원문 초록
Training embodied agents in simulation has become mainstream for the embodied AI community. However, these agents often struggle when deployed in the physical world due to their inability to generalize to real-world environments. In this paper, we present Phone2Proc, a method that uses a 10-minute phone scan and conditional procedural generation to create a distribution of training scenes that are semantically similar to the target environment. The generated scenes are conditioned on the wall layout and arrangement of large objects from the scan, while also sampling lighting, clutter, surface textures, and instances of smaller objects with randomized placement and materials. Leveraging just a simple RGB camera, training with Phone2Proc shows massive improvements from 34.7% to 70.7% success rate in sim-to-real ObjectNav performance across a test suite of over 200 trials in diverse real-world environments, including homes, offices, and RoboTHOR. Furthermore, Phone2Proc's diverse distribution of generated scenes makes agents remarkably robust to changes in the real world, such as human movement, object rearrangement, lighting changes, or clutter.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
PARSEC: Preference Adaptation for Robotic Object Rearrangement from Scene Context
ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Tasks
3DGS-CD: 3D Gaussian Splatting-based Change Detection for Physical Object Rearrangement
CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangement
Reinforcement Learning via Auxiliary Task Distillation
Grasp, See, and Place: Efficient Unknown Object Rearrangement with Policy Structure Prior
토론
이 논문과 연결된 포럼 글이 아직 없습니다.