Spatial Transport Optimization by Repositioning Attention Map for Training-Free Text-to-Image Synthesis
원문 초록
Diffusion-based text-to-image (T2I) models have recently excelled in high-quality image generation, particularly in a training-free manner, enabling cost-effective adaptability and generalization across diverse tasks. However, while the existing methods have been continuously focusing on several challenges, such as "missing objects" and "mismatched attributes," another critical issue of "mislocated objects" remains where generated spatial positions fail to align with text prompts. Surprisingly, ensuring such seemingly basic functionality remains challenging in popular T2I models due to the inherent difficulty of imposing explicit spatial guidance via text forms. To address this, we propose STORM (Spatial Transport Optimization by Repositioning Attention Map), a novel training-free approach for spatially coherent T2I synthesis. STORM employs Spatial Transport Optimization (STO), rooted in optimal transport theory, to dynamically adjust object attention maps for precise spatial adherence, supported by a Spatial Transport (ST) Cost function that enhances spatial understanding. Our analysis shows that integrating spatial awareness is most effective in the early denoising stages, while later phases refine details. Extensive experiments demonstrate that STORM surpasses existing methods, effectively mitigating mislocated objects while improving missing and mismatched attributes, setting a new benchmark for spatial alignment in T2I synthesis.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Noise Conditional Variational Score Distillation
fastWDM3D: Fast and Accurate 3D Healthy Tissue Inpainting
DiffusionBlocks: Blockwise Training for Generative Models via Score-Based Diffusion
Cost-Aware Routing for Efficient Text-To-Image Generation
High-resolution efficient image generation from WiFi CSI using a pretrained latent diffusion model
Stochastic Solutions for Simultaneous Seismic Data Denoising and Reconstruction via Score-based Generative Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.