L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors
원문 초록
Language-based colorization produces plausible and visually pleasing colors under the guidance of user-friendly natural language descriptions. Previous methods implicitly assume that users provide comprehensive color descriptions for most of the objects in the image, which leads to suboptimal performance. In this paper, we propose a unified model to perform language-based colorization with any-level descriptions. We leverage the pretrained cross-modality generative model for its robust language understanding and rich color priors to handle the inherent ambiguity of any-level descriptions. We further design modules to align with input conditions to preserve local spatial structures and prevent the ghosting effect. With the proposed novel sampling strategy, our model achieves instance-aware colorization in diverse and complex scenarios. Extensive experimental results demonstrate our advantages of effectively handling any-level descriptions and outperforming both language-based and automatic colorization methods. The code and pretrained models are available at: https://github.com/changzheng123/L-CAD.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
SSIMBaD: Sigma Scaling with SSIM-Guided Balanced Diffusion for AnimeFace Colorization
Leveraging the Powerful Attention of a Pre-trained Diffusion Model for Exemplar-based Image Colorization
Digital Staining with Knowledge Distillation: A Unified Framework for Unpaired and Paired-But-Misaligned Data
ColorizeDiffusion v2: Enhancing Reference-based Sketch Colorization Through Separating Utilities
A GAN-Enhanced Deep Learning Framework for Rooftop Detection from Historical Aerial Imagery
Language-based Image Colorization: A Benchmark and Beyond
토론
이 논문과 연결된 포럼 글이 아직 없습니다.