AGQA: A Benchmark for Compositional Spatio-Temporal Reasoning
원문 초록
Visual events are a composition of temporal actions involving actors spatially interacting with objects. When developing computer vision models that can reason about compositional spatio-temporal events, we need benchmarks that can analyze progress and uncover shortcomings. Existing video question answering benchmarks are useful, but they often conflate multiple sources of error into one accuracy metric and have strong biases that models can exploit, making it difficult to pinpoint model weaknesses. We present Action Genome Question Answering (AGQA), a new benchmark for compositional spatio-temporal reasoning. AGQA contains $192M$ unbalanced question answer pairs for $9.6K$ videos. We also provide a balanced subset of $3.9M$ question answer pairs, $3$ orders of magnitude larger than existing benchmarks, that minimizes bias by balancing the answer distributions and types of question structures. Although human evaluators marked $86.02\%$ of our question-answer pairs as correct, the best model achieves only $47.74\%$ accuracy. In addition, AGQA introduces multiple training/test splits to test for various reasoning abilities, including generalization to novel compositions, to indirect references, and to more compositional steps. Using AGQA, we evaluate modern visual reasoning systems, demonstrating that the best models barely perform better than non-visual baselines exploiting linguistic biases and that none of the existing models generalize to novel compositions unseen during training.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Decoupled Seg Tokens Make Stronger Reasoning Video Segmenter and Grounder
LLaVA-Scissor: Token Compression with Semantic Connected Components for Video LLMs
video-SALMONN 2: Captioning-Enhanced Audio-Visual Large Language Models
CausalVQA: A Physically Grounded Causal Reasoning Benchmark for Video Models
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
Looking Beyond Visible Cues: Implicit Video Question Answering via Dual-Clue Reasoning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.