Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation
원문 초록
Text-to-image models are powerful for producing high-quality images based on given text prompts, but crafting these prompts often requires specialized vocabulary. To address this, existing methods train rewriting models with supervision from large amounts of manually annotated data and trained aesthetic assessment models. To alleviate the dependence on data scale for model training and the biases introduced by trained models, we propose a novel prompt optimization framework, designed to rephrase a simple user prompt into a sophisticated prompt to a text-to-image model. Specifically, we employ the large vision language models (LVLMs) as the solver to rewrite the user prompt, and concurrently, employ LVLMs as a reward model to score the aesthetics and alignment of the images generated by the optimized prompt. Instead of laborious human feedback, we exploit the prior knowledge of the LVLM to provide rewards, i.e., AI feedback. Simultaneously, the solver and the reward model are unified into one model and iterated in reinforcement learning to achieve self-improvement by giving a solution and judging itself. Results on two popular datasets demonstrate that our method outperforms other strong competitors.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
CharaConsist: Fine-Grained Consistent Character Generation
NeoBabel: A Multilingual Open Tower for Visual Generation
Ovis-U1 Technical Report
Rethink Sparse Signals for Pose-guided Text-to-image Generation
XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulation
ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.