arXiv:2610.17902v1 cs.CV

Visual Dependence Does Not Imply Answerability: A Systematic Analysis of Answerability Signals in Frozen InstructBLIP

Minjun Kim, Hyunmin Song, Changwon Lim

제출 2026-10-13

저자 비고

arXiv Comments

10 pages, 3 figures, 5 tables

원문 초록

Visual question answering models often produce plausible responses even when the available visual evidence is insufficient to support a valid answer. Although prior work has introduced benchmarks and abstention mechanisms for unanswerable visual questions, it remains unclear which signals derived from a frozen vision-language model reliably indicate answerability on unseen image-question pairs. We systematically evaluate answerability signals on a stratified 10K subset of UNK-VQA using frozen InstructBLIP, covering predictive uncertainty, cross-modal alignment, visual grounding, visual dependence, internal representations, and learned probes. Answerability discrimination remains weak, with test AUROC largely confined to 0.50-0.54. CLIP alignment yields the highest point estimate but does not significantly outperform generation confidence under paired bootstrap analysis. Targeted removal of question-relevant visual evidence causes substantially larger changes in answer confidence than same-size random masking, yet intervention sensitivity provides essentially no answerability discrimination. Increasing probe capacity, combining multiple signals, and adding reason-aware supervision do not provide robust improvements, while post-freeze cross-validation yields similarly weak discrimination. Selective-answering analysis shows limited usable coverage under low-risk operating criteria. Failure-mode analysis shows that the relative utility of individual signals varies across perturbation mechanisms. Taken together, these results show that, on our UNK-VQA subset with frozen InstructBLIP, visual dependence, uncertainty, semantic alignment, and static representations are not robust standalone proxies for answerability. Our findings emphasize the need to distinguish a model's reliance on visual evidence from whether that evidence is sufficient to support an answer.

제출 이력

확인된 버전만 표시합니다. 전체 제출 이력은 arXiv 원문에서 확인하세요.

  1. arXiv:2610.17902v1

함께 살펴볼 논문

공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.

공통 태그로 연결된 공개 논문이 아직 없습니다.

토론

이 논문과 연결된 포럼 글이 아직 없습니다.