Enhancing Question Answering on Charts Through Effective Pre-training Tasks
원문 초록
To completely understand a document, the use of textual information is not enough. Understanding visual cues, such as layouts and charts, is also required. While the current state-of-the-art approaches for document understanding (both OCR-based and OCR-free) work well, a thorough analysis of their capabilities and limitations has not yet been performed. Therefore, in this work, we addresses the limitation of current VisualQA models when applied to charts and plots. To investigate shortcomings of the state-of-the-art models, we conduct a comprehensive behavioral analysis, using ChartQA as a case study. Our findings indicate that existing models particularly underperform in answering questions related to the chart's structural and visual context, as well as numerical information. To address these issues, we propose three simple pre-training tasks that enforce the existing model in terms of both structural-visual knowledge, as well as its understanding of numerical questions. We evaluate our pre-trained model (called MatCha-v2) on three chart datasets - both extractive and abstractive question datasets - and observe that it achieves an average improvement of 1.7% over the baseline model.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
DrishtiKon: Multi-Granular Visual Grounding for Text-Rich Document Images
PaddleOCR 3.0 Technical Report
SimpleDoc: Multi-Modal Document Understanding with Dual-Cue Page Retrieval and Iterative Refinement
Synthetic Document Question Answering in Hungarian
A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trends
Seeing is Believing? Mitigating OCR Hallucinations in Multimodal Large Language Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.