CoMPAS3D: A Dataset and Benchmark for Interactive Motion

원문 초록

Socially interactive humanoids aim to interact with humans through their bodies, adapting in real time to a partner's movement, intent, and abilities. This requires models that understand both how bodies move and what their movements mean. Yet, existing evaluation frameworks for interactive motion generation do not assess whether generated reactive motions are valid or appropriate within a shared movement vocabulary. Existing frameworks rely on kinematic metrics such as FID and beat alignment that cannot capture this semantic property, while improvised interaction datasets lack fine-grained move annotations. Inspired by computational evaluations of games such as Go and Chess, we propose Salsa as an embodied interaction evaluation domain. Salsa is interactive, dyadic, and governed by a move vocabulary and community-validated judging criteria covering timing, musicality, technique, difficulty, partnering, and originality. We present CoMPAS3D, a motion capture dataset of improvised partner salsa dance and an evaluation framework covering kinematic quality, two objective metrics (move legibility and proficiency appropriateness), and six competition-based subjective dimensions. The dataset comprises 3 hours of improvisation by 18 dancers spanning beginner, intermediate, and professional skill levels, with over 2,800 expert-annotated segments covering move types, execution errors, and stylistic elements. We define three benchmark tasks: move classification, proficiency estimation, and follower generation. Fine-tuned vision-language models perform well on ground-truth sequences and correlate with subjective expert judgments, suggesting their promise as measurement tools. Applied to Duolando and InterGen, our objective metrics reveal a large gap between ground-truth and generated dances, corroborated by expert human evaluations across six judging dimensions.