Neural Volumetric Memory for Visual Locomotion Control
원문 초록
Legged robots have the potential to expand the reach of autonomy beyond paved roads. In this work, we consider the difficult problem of locomotion on challenging terrains using a single forward-facing depth camera. Due to the partial observability of the problem, the robot has to rely on past observations to infer the terrain currently beneath it. To solve this problem, we follow the paradigm in computer vision that explicitly models the 3D geometry of the scene and propose Neural Volumetric Memory (NVM), a geometric memory architecture that explicitly accounts for the SE(3) equivariance of the 3D world. NVM aggregates feature volumes from multiple camera views by first bringing them back to the ego-centric frame of the robot. We test the learned visual-locomotion policy on a physical robot and show that our approach, which explicitly introduces geometric priors during training, offers superior performance than more na\"ive methods. We also include ablation studies and show that the representations stored in the neural volumetric memory capture sufficient geometric information to reconstruct the scene. Our project page with videos is https://rchalyang.github.io/NVM .
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
SparseGS: Sparse View Synthesis using 3D Gaussian Splatting
Feed-Forward SceneDINO for Unsupervised Semantic Scene Completion
Dense 3D Displacement Estimation for Landslide Monitoring via Fusion of TLS Point Clouds and Embedded RGB Images
ScaleLSD: Scalable Deep Line Segment Detection Streamlined
GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control
Recollection from Pensieve: Novel View Synthesis via Learning from Uncalibrated Videos
토론
이 논문과 연결된 포럼 글이 아직 없습니다.