Better Together: Unified Motion Capture and 3D Avatar Reconstruction
원문 초록
We present Better Together, a method that simultaneously solves the human pose estimation problem while reconstructing a photorealistic 3D human avatar from multi-view videos. While prior art usually solves these problems separately, we argue that joint optimization of skeletal motion with a 3D renderable body model brings synergistic effects, i.e. yields more precise motion capture and improved visual quality of real-time rendering of avatars. To achieve this, we introduce a novel animatable avatar with 3D Gaussians rigged on a personalized mesh and propose to optimize the motion sequence with time-dependent MLPs that provide accurate and temporally consistent pose estimates. We first evaluate our method on highly challenging yoga poses and demonstrate state-of-the-art accuracy on multi-view human pose estimation, reducing error by 35% on body joints and 45% on hand joints compared to keypoint-based methods. At the same time, our method significantly boosts the visual quality of animatable avatars (+2dB PSNR on novel view synthesis) on diverse challenging subjects.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Recollection from Pensieve: Novel View Synthesis via Learning from Uncalibrated Videos
EndoFlow-SLAM: Real-Time Endoscopic SLAM with Flow-Constrained Gaussian Splatting
Princeton365: A Diverse Dataset with Accurate Camera Pose
Hierarchical Scoring with 3D Gaussian Splatting for Instance Image-Goal Navigation
SurGSplat: Progressive Geometry-Constrained Gaussian Splatting for Surgical Scene Reconstruction
On-the-fly Reconstruction for Large-Scale Novel View Synthesis from Unposed Images
토론
이 논문과 연결된 포럼 글이 아직 없습니다.