DiffuVST: Narrating Fictional Scenes with Global-History-Guided Denoising Models
원문 초록
Recent advances in image and video creation, especially AI-based image synthesis, have led to the production of numerous visual scenes that exhibit a high level of abstractness and diversity. Consequently, Visual Storytelling (VST), a task that involves generating meaningful and coherent narratives from a collection of images, has become even more challenging and is increasingly desired beyond real-world imagery. While existing VST techniques, which typically use autoregressive decoders, have made significant progress, they suffer from low inference speed and are not well-suited for synthetic scenes. To this end, we propose a novel diffusion-based system DiffuVST, which models the generation of a series of visual descriptions as a single conditional denoising process. The stochastic and non-autoregressive nature of DiffuVST at inference time allows it to generate highly diverse narratives more efficiently. In addition, DiffuVST features a unique design with bi-directional text history guidance and multimodal adapter modules, which effectively improve inter-sentence coherence and image-to-text fidelity. Extensive experiments on the story generation task covering four fictional visual-story datasets demonstrate the superiority of DiffuVST over traditional autoregressive models in terms of both text quality and inference speed.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
fastWDM3D: Fast and Accurate 3D Healthy Tissue Inpainting
Synthesizing Reality: Leveraging the Generative AI-Powered Platform Midjourney for Construction Worker Detection
AdaptaGen: Domain-Specific Image Generation through Hierarchical Semantic Optimization Framework
Beyond Blur: A Fluid Perspective on Generative Diffusion Models
AI's Blind Spots: Geographic Knowledge and Diversity Deficit in Generated Urban Scenario
Cost-Aware Routing for Efficient Text-To-Image Generation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.