Answering Visual What-If Questions: From Actions to Predicted Scene Descriptions
원문 초록
In-depth scene descriptions and question answering tasks have greatly increased the scope of today's definition of scene understanding. While such tasks are in principle open ended, current formulations primarily focus on describing only the current state of the scenes under consideration. In contrast, in this paper, we focus on the future states of the scenes which are also conditioned on actions. We posit this as a question answering task, where an answer has to be given about a future scene state, given observations of the current scene, and a question that includes a hypothetical action. Our solution is a hybrid model which integrates a physics engine into a question answering architecture in order to anticipate future scene states resulting from object-object interactions caused by an action. We demonstrate first results on this challenging new problem and compare to baselines, where we outperform fully data-driven end-to-end learning approaches.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
City-VLM: Towards Multidomain Perception Scene Understanding via Multimodal Incomplete Learning
Scene-R1: Video-Grounded Large Language Models for 3D Scene Reasoning without 3D Annotations
PhyBlock: A Progressive Benchmark for Physical Understanding and Planning via 3D Block Assembly
Does Your 3D Encoder Really Work? When Pretrain-SFT from 2D VLMs Meets 3D VLMs
Learning Sparsity for Effective and Efficient Music Performance Question Answering
From Roots to Rewards: Dynamic Tree Reasoning with RL
토론
이 논문과 연결된 포럼 글이 아직 없습니다.