Control3D: Towards Controllable Text-to-3D Generation
원문 초록
Recent remarkable advances in large-scale text-to-image diffusion models have inspired a significant breakthrough in text-to-3D generation, pursuing 3D content creation solely from a given text prompt. However, existing text-to-3D techniques lack a crucial ability in the creative process: interactively control and shape the synthetic 3D contents according to users' desired specifications (e.g., sketch). To alleviate this issue, we present the first attempt for text-to-3D generation conditioning on the additional hand-drawn sketch, namely Control3D, which enhances controllability for users. In particular, a 2D conditioned diffusion model (ControlNet) is remoulded to guide the learning of 3D scene parameterized as NeRF, encouraging each view of 3D scene aligned with the given text prompt and hand-drawn sketch. Moreover, we exploit a pre-trained differentiable photo-to-sketch model to directly estimate the sketch of the rendered image over synthetic 3D scene. Such estimated sketch along with each sampled view is further enforced to be geometrically consistent with the given sketch, pursuing better controllable text-to-3D generation. Through extensive experiments, we demonstrate that our proposal can generate accurate and faithful 3D scenes that align closely with the input text prompts and sketches.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Generating Physically Stable and Buildable LEGO Designs from Text
Apply Hierarchical-Chain-of-Generation to Complex Attributes Text-to-3D Generation
Text-based Animatable 3D Avatars with Morphable Model Alignment
ConsDreamer: Advancing Multi-View Consistency for Zero-Shot Text-to-3D Generation
RewardSDS: Aligning Score Distillation via Reward-Weighted Sampling
DreamArt: Generating Interactable Articulated Objects from a Single Image
토론
이 논문과 연결된 포럼 글이 아직 없습니다.