Diffusion Implicit Policy for Unpaired Scene-aware Motion Synthesis
원문 초록
Human motion generation is a long-standing problem, and scene-aware motion synthesis has been widely researched recently due to its numerous applications. Prevailing methods rely heavily on paired motion-scene data whose quantity is limited. Meanwhile, it is difficult to generalize to diverse scenes when trained only on a few specific ones. Thus, we propose a unified framework, termed Diffusion Implicit Policy (DIP), for scene-aware motion synthesis, where paired motion-scene data are no longer necessary. In this framework, we disentangle human-scene interaction from motion synthesis during training and then introduce an interaction-based implicit policy into motion diffusion during inference. Synthesized motion can be derived through iterative diffusion denoising and implicit policy optimization, thus motion naturalness and interaction plausibility can be maintained simultaneously. The proposed implicit policy optimizes the intermediate noised motion in a GAN Inversion manner to maintain motion continuity and control keyframe poses though the ControlNet branch and motion inpainting. For long-term motion synthesis, we introduce motion blending for stable transitions between multiple sub-tasks, where motions are fused in rotation power space and translation linear space. The proposed method is evaluated on synthesized scenes with ShapeNet furniture, and real scenes from PROX and Replica. Results show that our framework presents better motion naturalness and interaction plausibility than cutting-edge methods. This also indicates the feasibility of utilizing the DIP for motion synthesis in more general tasks and versatile scenes. https://jingyugong.github.io/DiffusionImplicitPolicy/
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
PlanMoGPT: Flow-Enhanced Progressive Planning for Text to Motion Synthesis
Motion-R1: Chain-of-Thought Reasoning and Reinforcement Learning for Human Motion Generation
MotionPro: A Precise Motion Controller for Image-to-Video Generation
Cross-Conditioned Recurrent Networks for Long-Term Synthesis of Inter-Person Human Motion Interactions
Refining Few-Step Text-to-Multiview Diffusion via Reinforcement Learning
On Designing Diffusion Autoencoders for Efficient Generation and Representation Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.