Hierarchical Motion Understanding via Motion Programs
원문 초록
Current approaches to video analysis of human motion focus on raw pixels or keypoints as the basic units of reasoning. We posit that adding higher-level motion primitives, which can capture natural coarser units of motion such as backswing or follow-through, can be used to improve downstream analysis tasks. This higher level of abstraction can also capture key features, such as loops of repeated primitives, that are currently inaccessible at lower levels of representation. We therefore introduce Motion Programs, a neuro-symbolic, program-like representation that expresses motions as a composition of high-level primitives. We also present a system for automatically inducing motion programs from videos of human motion and for leveraging motion programs in video synthesis. Experiments show that motion programs can accurately describe a diverse set of human motions and the inferred programs contain semantically meaningful motion primitives, such as arm swings and jumping jacks. Our representation also benefits downstream tasks such as video interpolation and video prediction and outperforms off-the-shelf models. We further demonstrate how these programs can detect diverse kinds of repetitive motion and facilitate interactive video editing.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Learning 3D Particle-based Simulators from RGB-D Videos
Epona: Autoregressive Diffusion World Model for Autonomous Driving
Causally Steered Diffusion for Automated Video Counterfactual Generation
FADE: Frequency-Aware Diffusion Model Factorization for Video Editing
Zero-to-Hero: Zero-Shot Initialization Empowering Reference-Based Video Appearance Editing
Video Editing for Audio-Visual Dubbing
토론
이 논문과 연결된 포럼 글이 아직 없습니다.