PanoFree: Tuning-Free Holistic Multi-view Image Generation with Cross-view Self-Guidance
원문 초록
Immersive scene generation, notably panorama creation, benefits significantly from the adaptation of large pre-trained text-to-image (T2I) models for multi-view image generation. Due to the high cost of acquiring multi-view images, tuning-free generation is preferred. However, existing methods are either limited to simple correspondences or require extensive fine-tuning to capture complex ones. We present PanoFree, a novel method for tuning-free multi-view image generation that supports an extensive array of correspondences. PanoFree sequentially generates multi-view images using iterative warping and inpainting, addressing the key issues of inconsistency and artifacts from error accumulation without the need for fine-tuning. It improves error accumulation by enhancing cross-view awareness and refines the warping and inpainting processes via cross-view guidance, risky area estimation and erasing, and symmetric bidirectional guided generation for loop closure, alongside guidance-based semantic and density control for scene structure preservation. In experiments on Planar, 360{\deg}, and Full Spherical Panoramas, PanoFree demonstrates significant error reduction, improves global consistency, and boosts image quality without extra fine-tuning. Compared to existing methods, PanoFree is up to 5x more efficient in time and 3x more efficient in GPU memory usage, and maintains superior diversity of results (2x better in our user study). PanoFree offers a viable alternative to costly fine-tuning or the use of additional pre-trained models. Project website at https://panofree.github.io/.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulation
ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generation
Instella-T2I: Pushing the Limits of 1D Discrete Latent Space Image Generation
Beyond Blur: A Fluid Perspective on Generative Diffusion Models
DreamCube: 3D Panorama Generation via Multi-plane Synchronization
VideoMAR: Autoregressive Video Generatio with Continuous Tokens
토론
이 논문과 연결된 포럼 글이 아직 없습니다.