GVDIFF: Grounded Text-to-Video Generation with Diffusion Models
원문 초록
In text-to-video (T2V) generation, significant attention has been directed toward its development, yet unifying discrete and continuous grounding conditions in T2V generation remains under-explored. This paper proposes a Grounded text-to-Video generation framework, termed GVDIFF. First, we inject the grounding condition into the self-attention through an uncertainty-based representation to explicitly guide the focus of the network. Second, we introduce a spatial-temporal grounding layer that connects the grounding condition with target objects and enables the model with the grounded generation capacity in the spatial-temporal domain. Third, our dynamic gate network adaptively skips the redundant grounding process to selectively extract grounding information and semantics while improving efficiency. We extensively evaluate the grounded generation capacity of GVDIFF and demonstrate its versatility in applications, including long-range video generation, sequential prompts, and object-specific editing.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
VCapsBench: A Large-scale Fine-grained Benchmark for Video Caption Quality Evaluation
InfLVG: Reinforce Inference-Time Consistent Long Video Generation with GRPO
LOVE: Benchmarking and Evaluating Text-to-Video Generation and Video-to-Text Interpretation
On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devices
VPO: Aligning Text-to-Video Generation Models with Prompt Optimization
RecTable: Fast Modeling Tabular Data with Rectified Flow
토론
이 논문과 연결된 포럼 글이 아직 없습니다.