Steering Rectified Flow Models in the Vector Field for Controlled Image Generation
원문 초록
Diffusion models (DMs) excel in photorealism, image editing, and solving inverse problems, aided by classifier-free guidance and image inversion techniques. However, rectified flow models (RFMs) remain underexplored for these tasks. Existing DM-based methods often require additional training, lack generalization to pretrained latent models, underperform, and demand significant computational resources due to extensive backpropagation through ODE solvers and inversion processes. In this work, we first develop a theoretical and empirical understanding of the vector field dynamics of RFMs in efficiently guiding the denoising trajectory. Our findings reveal that we can navigate the vector field in a deterministic and gradient-free manner. Utilizing this property, we propose FlowChef, which leverages the vector field to steer the denoising trajectory for controlled image generation tasks, facilitated by gradient skipping. FlowChef is a unified framework for controlled image generation that, for the first time, simultaneously addresses classifier guidance, linear inverse problems, and image editing without the need for extra training, inversion, or intensive backpropagation. Finally, we perform extensive evaluations and show that FlowChef significantly outperforms baselines in terms of performance, memory, and time requirements, achieving new state-of-the-art results. Project Page: \url{https://flowchef.github.io}.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Noise Conditional Variational Score Distillation
fastWDM3D: Fast and Accurate 3D Healthy Tissue Inpainting
DiffusionBlocks: Blockwise Training for Generative Models via Score-Based Diffusion
Cost-Aware Routing for Efficient Text-To-Image Generation
High-resolution efficient image generation from WiFi CSI using a pretrained latent diffusion model
Stochastic Solutions for Simultaneous Seismic Data Denoising and Reconstruction via Score-based Generative Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.