DiffBrush:Just Painting the Art by Your Hands
원문 초록
The rapid development of image generation and editing algorithms in recent years has enabled ordinary user to produce realistic images. However, the current AI painting ecosystem predominantly relies on text-driven diffusion models (T2I), which pose challenges in accurately capturing user requirements. Furthermore, achieving compatibility with other modalities incurs substantial training costs. To this end, we introduce DiffBrush, which is compatible with T2I models and allows users to draw and edit images. By manipulating and adapting the internal representation of the diffusion model, DiffBrush guides the model-generated images to converge towards the user's hand-drawn sketches for user's specific needs without additional training. DiffBrush achieves control over the color, semantic, and instance of objects in images by continuously guiding the latent and instance-level attention map during the denoising process of the diffusion model. Besides, we propose a latent regeneration, which refines the randomly sampled noise in the diffusion model, obtaining a better image generation layout. Finally, users only need to roughly draw the mask of the instance (acceptable colors) on the canvas, DiffBrush can naturally generate the corresponding instance at the corresponding location.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Noise Conditional Variational Score Distillation
fastWDM3D: Fast and Accurate 3D Healthy Tissue Inpainting
DiffusionBlocks: Blockwise Training for Generative Models via Score-Based Diffusion
Cost-Aware Routing for Efficient Text-To-Image Generation
High-resolution efficient image generation from WiFi CSI using a pretrained latent diffusion model
Stochastic Solutions for Simultaneous Seismic Data Denoising and Reconstruction via Score-based Generative Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.