I4VGen: Image as Free Stepping Stone for Text-to-Video Generation
원문 초록
Text-to-video generation has trailed behind text-to-image generation in terms of quality and diversity, primarily due to the inherent complexities of spatio-temporal modeling and the limited availability of video-text datasets. Recent text-to-video diffusion models employ the image as an intermediate step, significantly enhancing overall performance but incurring high training costs. In this paper, we present I4VGen, a novel video diffusion inference pipeline to leverage advanced image techniques to enhance pre-trained text-to-video diffusion models, which requires no additional training. Instead of the vanilla text-to-video inference pipeline, I4VGen consists of two stages: anchor image synthesis and anchor image-augmented text-to-video synthesis. Correspondingly, a simple yet effective generation-selection strategy is employed to achieve visually-realistic and semantically-faithful anchor image, and an innovative noise-invariant video score distillation sampling (NI-VSDS) is developed to animate the image to a dynamic video by distilling motion knowledge from video diffusion models, followed by a video regeneration process to refine the video. Extensive experiments show that the proposed method produces videos with higher visual realism and textual fidelity. Furthermore, I4VGen also supports being seamlessly integrated into existing image-to-video diffusion models, thereby improving overall video quality.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
CharaConsist: Fine-Grained Consistent Character Generation
NeoBabel: A Multilingual Open Tower for Visual Generation
Ovis-U1 Technical Report
Rethink Sparse Signals for Pose-guided Text-to-image Generation
XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulation
ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.