SSDNeRF: Semantic Soft Decomposition of Neural Radiance Fields
원문 초록
Neural Radiance Fields (NeRFs) encode the radiance in a scene parameterized by the scene's plenoptic function. This is achieved by using an MLP together with a mapping to a higher-dimensional space, and has been proven to capture scenes with a great level of detail. Naturally, the same parameterization can be used to encode additional properties of the scene, beyond just its radiance. A particularly interesting property in this regard is the semantic decomposition of the scene. We introduce a novel technique for semantic soft decomposition of neural radiance fields (named SSDNeRF) which jointly encodes semantic signals in combination with radiance signals of a scene. Our approach provides a soft decomposition of the scene into semantic parts, enabling us to correctly encode multiple semantic classes blending along the same direction -- an impossible feat for existing methods. Not only does this lead to a detailed, 3D semantic representation of the scene, but we also show that the regularizing effects of the MLP used for encoding help to improve the semantic representation. We show state-of-the-art segmentation and reconstruction results on a dataset of common objects and demonstrate how the proposed approach can be applied for high quality temporally consistent video editing and re-compositing on a dataset of casually captured selfie videos.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Causally Steered Diffusion for Automated Video Counterfactual Generation
FADE: Frequency-Aware Diffusion Model Factorization for Video Editing
Zero-to-Hero: Zero-Shot Initialization Empowering Reference-Based Video Appearance Editing
Video Editing for Audio-Visual Dubbing
DC-SAM: In-Context Segment Anything in Images and Videos via Dual Consistency
FlashDepth: Real-time Streaming Video Depth Estimation at 2K Resolution
토론
이 논문과 연결된 포럼 글이 아직 없습니다.