MotionBooth: Motion-Aware Customized Text-to-Video Generation
원문 초록
In this work, we present MotionBooth, an innovative framework designed for animating customized subjects with precise control over both object and camera movements. By leveraging a few images of a specific object, we efficiently fine-tune a text-to-video model to capture the object's shape and attributes accurately. Our approach presents subject region loss and video preservation loss to enhance the subject's learning performance, along with a subject token cross-attention loss to integrate the customized subject with motion control signals. Additionally, we propose training-free techniques for managing subject and camera motions during inference. In particular, we utilize cross-attention map manipulation to govern subject motion and introduce a novel latent shift module for camera movement control as well. MotionBooth excels in preserving the appearance of subjects while simultaneously controlling the motions in generated videos. Extensive quantitative and qualitative evaluations demonstrate the superiority and effectiveness of our method. Our project page is at https://jianzongwu.github.io/projects/motionbooth
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
VCapsBench: A Large-scale Fine-grained Benchmark for Video Caption Quality Evaluation
InfLVG: Reinforce Inference-Time Consistent Long Video Generation with GRPO
LOVE: Benchmarking and Evaluating Text-to-Video Generation and Video-to-Text Interpretation
On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devices
VPO: Aligning Text-to-Video Generation Models with Prompt Optimization
RecTable: Fast Modeling Tabular Data with Rectified Flow
토론
이 논문과 연결된 포럼 글이 아직 없습니다.