Differentiable Blocks World: Qualitative 3D Decomposition by Rendering Primitives
원문 초록
Given a set of calibrated images of a scene, we present an approach that produces a simple, compact, and actionable 3D world representation by means of 3D primitives. While many approaches focus on recovering high-fidelity 3D scenes, we focus on parsing a scene into mid-level 3D representations made of a small set of textured primitives. Such representations are interpretable, easy to manipulate and suited for physics-based simulations. Moreover, unlike existing primitive decomposition methods that rely on 3D input data, our approach operates directly on images through differentiable rendering. Specifically, we model primitives as textured superquadric meshes and optimize their parameters from scratch with an image rendering loss. We highlight the importance of modeling transparency for each primitive, which is critical for optimization and also enables handling varying numbers of primitives. We show that the resulting textured primitives faithfully reconstruct the input images and accurately model the visible 3D points, while providing amodal shape completions of unseen object regions. We compare our approach to the state of the art on diverse scenes from DTU, and demonstrate its robustness on real-life captures from BlendedMVS and Nerfstudio. We also showcase how our results can be used to effortlessly edit a scene or perform physical simulations. Code and video results are available at https://www.tmonnier.com/DBW .
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Dynamic Diffusion Schrödinger Bridge in Astrophysical Observational Inversions
Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systems
Adversarial Manhole: Challenging Monocular Depth Estimation and Semantic Segmentation Models with Patch Attack
Efficient Probabilistic Modeling of Crystallization at Mesoscopic Scale
Interpolating neural network: A novel unification of machine learning and interpolation theory
Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer
토론
이 논문과 연결된 포럼 글이 아직 없습니다.