FLAME-in-NeRF : Neural control of Radiance Fields for Free View Face Animation
원문 초록
This paper presents a neural rendering method for controllable portrait video synthesis. Recent advances in volumetric neural rendering, such as neural radiance fields (NeRF), has enabled the photorealistic novel view synthesis of static scenes with impressive results. However, modeling dynamic and controllable objects as part of a scene with such scene representations is still challenging. In this work, we design a system that enables both novel view synthesis for portrait video, including the human subject and the scene background, and explicit control of the facial expressions through a low-dimensional expression representation. We leverage the expression space of a 3D morphable face model (3DMM) to represent the distribution of human facial expressions, and use it to condition the NeRF volumetric function. Furthermore, we impose a spatial prior brought by 3DMM fitting to guide the network to learn disentangled control for scene appearance and facial actions. We demonstrate the effectiveness of our method on free view synthesis of portrait videos with expression controls. To train a scene, our method only requires a short video of a subject captured by a mobile device.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
R3eVision: A Survey on Robust Rendering, Restoration, and Enhancement for 3D Low-Level Vision
GoLF-NRT: Integrating Global Context and Local Geometry for Few-Shot View Synthesis
SLAM&Render: A Benchmark for the Intersection Between Neural Rendering, Gaussian Splatting and SLAM
ProJo4D: Progressive Joint Optimization for Sparse-View Inverse Physics Estimation
UnMix-NeRF: Spectral Unmixing Meets Neural Radiance Fields
Self-Supervised Multimodal NeRF for Autonomous Driving
토론
이 논문과 연결된 포럼 글이 아직 없습니다.