Unimotion: Unifying 3D Human Motion Synthesis and Understanding
원문 초록
We introduce Unimotion, the first unified multi-task human motion model capable of both flexible motion control and frame-level motion understanding. While existing works control avatar motion with global text conditioning, or with fine-grained per frame scripts, none can do both at once. In addition, none of the existing works can output frame-level text paired with the generated poses. In contrast, Unimotion allows to control motion with global text, or local frame-level text, or both at once, providing more flexible control for users. Importantly, Unimotion is the first model which by design outputs local text paired with the generated poses, allowing users to know what motion happens and when, which is necessary for a wide range of applications. We show Unimotion opens up new applications: 1.) Hierarchical control, allowing users to specify motion at different levels of detail, 2.) Obtaining motion text descriptions for existing MoCap data or YouTube videos 3.) Allowing for editability, generating motion from text, and editing the motion via text edits. Moreover, Unimotion attains state-of-the-art results for the frame-level text-to-motion task on the established HumanML3D dataset. The pre-trained model and code are available available on our project page at https://coral79.github.io/uni-motion/.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
DeepGesture: A conversational gesture synthesis system based on emotions and semantics
VolumetricSMPL: A Neural Volumetric Body Model for Efficient Interactions, Contacts, and Collisions
DuetGen: Music Driven Two-Person Dance Generation via Hierarchical Masked Modeling
Generative AI for Character Animation: A Comprehensive Survey of Techniques, Applications, and Future Directions
Reinforcement learning-based motion imitation for physiologically plausible musculoskeletal motor control
AnyMoLe: Any Character Motion In-betweening Leveraging Video Diffusion Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.