데이터셋 탐색

Form Understanding in Noisy Scanned Documents (FUNSD) comprises 199 real, fully annotated, scanned forms. The documents are noisy and vary widely in appearance, making form understanding (FoUn) a challenging task. The proposed dataset can be used for various tasks, including text detection, optical character recognition, spatial layout analysis, and entity labeling/linking. Source: FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents Image source: https://guillaumejaume.github.io/FUNSD/

RFUND is a relabeled version of FUNSD and XFUND datasets, tackling the following issues in their original annotations: 1. Entity (block) level OCR results. Real-world OCR engines usually produce line-level results, while the annotations in FUNSD and XFUND are at the entity (block) level. Text lines within the same entity are aggregated and serialized in human reading order, simplifying the task scope and failing to reflect real-world challenges. 2. Inconsistent labelling granularity. In FUNSD, while most contents are annotated at the entity level, multi-line entities with first-line indentation are annotated separately, in which the first line is split out and the rest are aggregated. XFUND exhibits variable granularity in annotations, with some contents labelled at the entity level and others at the line level. Such inconsistent labelling standards can hinder model training. 3. Erroneous category annotations. Entities in FUNSD/XFUND are categorized as "header", "question", "answer" and "other". We observed that certain entities in both FUNSD and XFUND have category labels that differ from human understanding. In the following example, the answer entity “Client confirmed agreement ...” was labelled as other, while the other entity “CONFIDENTIAL” was labelled as the question.

EC-FUNSD is introduced in [arXiv:2402.02379] as a benchmark of semantic entity recognition (SER) and entity linking (EL), designed for the entity-centric robustness evaluation of pre-trained text-and-layout models (PTLMs). In practical applications of document intelligence, PTLMs (e.g. the LayoutLM series) generally serve as the encoder of document layouts, similar to the role played by pre-trained contextualized language models (e.g. the BERT series) in NLP tasks. Conventionally, information extraction (IE) ability of PTLMs is evaluated via SER and EL, esp. in a sequence-labeling manner. The performance of PTLMs on these tasks reflects the capacity of their layout embeddings to facilitate downstream IE tasks. However, the prevailing benchmarks do not fully conform to the aforementioned evaluation pipeline, thereby diminishing the reliability of the assessment. Take FUNSD as an example, its block-level annotation falsely couples the annotations of segment and entity, which does not adequately represent semantic-driven entities and hinder the fair evaluation. The propose of EC-FUNSD aims to provide a fair and unbiased evaluation benchmark of IE ability of PTLMs. The construction of this dataset includes the revision of layout and IE annotations from FUNSD. First, the original layout annotation of FUNSD is cleaned, and multiple-row blocks are split into row-wise segments. Second, the semantic entities are re-annotated together with their linking relationships, with the segment order preserved to ensure that each entity is represented as a continuous word span within layout, making the form of this dataset suitable for sequence-labeling models. The final dataset consists of 199 document samples including the image, layout annotation of segments and words, and labeled entities of 3 categories. For the detailed annotation process and statistics, please refer to the original paper.

We introduce FUNSD-r and CORD-r in Token Path Prediction, the revised VrD-NER datasets to reflect the real-world scenarios of NER on scanned VrDs. In FUNSD and CORD, segment layout annotations are aligned with labeled entities, which makes them not reflect the reading order issue of NER on scanned VrDs, and thus are unsuitable for evaluating current methods. In FUNSD-r and CORD-r, we automatically reannotate the layouts using PP-OCRv3 OCR system, and manually reannotate the named entities as word sequences based on the new layout annotations. Their segment layout annotations are aligned with real-world situations and entity mentions are labeled on words. The proposed FUNSD-r consists of 199 document samples including the image, layout annotation of segments and words, and labeled entities of 3 categories. For the detailed summary statistics, please refer to the original paper.