arXiv:2505.10743

IMAGE-ALCHEMY: Advancing subject fidelity in personalised text-to-image generation

Amritanshu Tiwari, Cherish Puniani, Kaustubh Sharma, Ojasva Nema

제출 2025-05-15

원문 초록

Recent advances in text-to-image diffusion models, particularly Stable Diffusion, have enabled the generation of highly detailed and semantically rich images. However, personalizing these models to represent novel subjects based on a few reference images remains challenging. This often leads to catastrophic forgetting, overfitting, or large computational overhead.We propose a two-stage pipeline that addresses these limitations by leveraging LoRA-based fine-tuning on the attention weights within the U-Net of the Stable Diffusion XL (SDXL) model. First, we use the unmodified SDXL to generate a generic scene by replacing the subject with its class label. Then, we selectively insert the personalized subject through a segmentation-driven image-to-image (Img2Img) pipeline that uses the trained LoRA weights.This framework isolates the subject encoding from the overall composition, thus preserving SDXL's broader generative capabilities while integrating the new subject in a high-fidelity manner. Our method achieves a DINO similarity score of 0.789 on SDXL, outperforming existing personalized text-to-image approaches.

함께 살펴볼 논문

공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.

arXiv:2506.23044공통 태그 Image Generation, Text-to-Image Generation, Text to Image Generation공식 코드 · Papers with Code

Ovis-U1 Technical Report

토론

이 논문과 연결된 포럼 글이 아직 없습니다.