TLControl: Trajectory and Language Control for Human Motion Synthesis
원문 초록
Controllable human motion synthesis is essential for applications in AR/VR, gaming and embodied AI. Existing methods often focus solely on either language or full trajectory control, lacking precision in synthesizing motions aligned with user-specified trajectories, especially for multi-joint control. To address these issues, we present TLControl, a novel method for realistic human motion synthesis, incorporating both low-level Trajectory and high-level Language semantics controls, through the integration of neural-based and optimization-based techniques. Specifically, we begin with training a VQ-VAE for a compact and well-structured latent motion space organized by body parts. We then propose a Masked Trajectories Transformer (MTT) for predicting a motion distribution conditioned on language and trajectory. Once trained, we use MTT to sample initial motion predictions given user-specified partial trajectories and text descriptions as conditioning. Finally, we introduce a test-time optimization to refine these coarse predictions for precise trajectory control, which offers flexibility by allowing users to specify various optimization goals and ensures high runtime efficiency. Comprehensive experiments show that TLControl significantly outperforms the state-of-the-art in trajectory accuracy and time efficiency, making it practical for interactive and high-quality animation generation.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
DeepGesture: A conversational gesture synthesis system based on emotions and semantics
VolumetricSMPL: A Neural Volumetric Body Model for Efficient Interactions, Contacts, and Collisions
DuetGen: Music Driven Two-Person Dance Generation via Hierarchical Masked Modeling
Generative AI for Character Animation: A Comprehensive Survey of Techniques, Applications, and Future Directions
Reinforcement learning-based motion imitation for physiologically plausible musculoskeletal motor control
AnyMoLe: Any Character Motion In-betweening Leveraging Video Diffusion Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.