arXiv:2609.34480v2 cs.CV

When Does an Image Determine the Answer? Benchmarking Visual Answerability across Charts and Scenes

Sungguk Cha, Mintae Kim, Youngsub Han, Byoung-Ki Jeon, Sangyeob Lee

제출 2026-11-13

원문 초록

Reliable visual question answering requires correct answers when evidence is sufficient and abstention when it is not. We introduce a benchmark of 3,000 question groups: 1,000 each from PlotQA charts, CLEVR rendered scenes, and GQA photographs. Each group keeps the question fixed across original, answer-preserving, and evidence-removing images; charts and CLEVR also include answer-changing edits. Models receive each image independently. A group succeeds only if every supported answer and every required abstention is correct. For charts and CLEVR, executable witnesses establish ambiguity under the declared task semantics: two admissible complete charts or scenes give different answers but yield the same masked observation. GQA scoring compares source labels with a permissive rule that also credits masked-image answers compatible with the full-image reference. Across 72,000 responses from six model configurations, the best complete-question success is 57.0% on charts, 43.5% on CLEVR, and 33.7% on GQA under source labels. On charts, the strongest configuration makes correct answer-or-abstain decisions on 96.2% of individual images and on all five images of 83.5% of questions. Requiring correct answers lowers success to 57.0%: in 26.5% of all chart questions, every decision is correct but at least one answer is wrong. Adding answer correctness also reverses a chart ranking based on decision accuracy. Evaluating all response obligations together reveals failures that per-image decision accuracy conceals.

제출 이력

확인된 버전만 표시합니다. 전체 제출 이력은 arXiv 원문에서 확인하세요.

  1. arXiv:2609.34480v2제출 시각 미확인

함께 살펴볼 논문

공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.

공통 태그로 연결된 공개 논문이 아직 없습니다.

토론

이 논문과 연결된 포럼 글이 아직 없습니다.