Token Classification

The MNIST database (Modified National Institute of Standards and Technology database) is a large collection of handwritten digits. It has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger NIST Special Database 3 (digits written by employees of the United States Census Bureau) and Special Database 1 (digits written by high school students) which contain monochrome images of handwritten digits. The digits have been size-normalized and centered in a fixed-size image. The original black and white (bilevel) images from NIST were size normalized to fit in a 20x20 pixel box while preserving their aspect ratio. The resulting images contain grey levels as a result of the anti-aliasing technique used by the normalization algorithm. the images were centered in a 28x28 image by computing the center of mass of the pixels, and translating the image so as to position this point at the center of the 28x28 field. Source: http://yann.lecun.com/exdb/mnist/ Image Source: https://en.wikipedia.org/wiki/MNIST_database#/media/File:MnistExamples.png

CoNLL-2003 is a named entity recognition dataset released as a part of CoNLL-2003 shared task: language-independent named entity recognition. The data consists of eight files covering two languages: English and German. For each of the languages there is a training file, a development file, a test file and a large file with unannotated data. The English data was taken from the Reuters Corpus. This corpus consists of Reuters news stories between August 1996 and August 1997. For the training and development set, ten days worth of data were taken from the files representing the end of August 1996. For the test set, the texts were from December 1996. The preprocessed raw data covers the month of September 1996. The text for the German data was taken from the ECI Multilingual Text Corpus. This corpus consists of texts in many languages. The portion of data that was used for this task, was extracted from the German newspaper Frankfurter Rundshau. All three of the training, development and test sets were taken from articles written in one week at the end of August 1992. The raw data were taken from the months of September to December 1992. | English data | Articles | Sentences | Tokens | LOC | MISC | ORG | PER | |-------------------|----------|-----------|---------|------|------|------|------| | Training set | 946 | 14,987 | 203,621 | 7140 | 3438 | 6321 | 6600 | | Development set | 216 | 3,466 | 51,362 | 1837 | 922 | 1341 | 1842 | | Test set | 231 | 3,684 | 46,435 | 1668 | 702 | 1661 | 1617 | Number of articles, sentences, tokens and entities (locations, miscellaneous, organizations, and persons) in English data files. | German data | Articles | Sentences | Tokens | LOC | MISC | ORG | PER | |-------------------|----------|-----------|---------|------|------|------|------| | Training set | 553 | 12,705 | 206,931 | 4363 | 2288 | 2427 | 2773 | | Development set | 201 | 3,068 | 51,444 | 1181 | 1010 | 1241 | 1401 | | Test set | 155 | 3,160 | 51,943 | 1035 | 670 | 773 | 1195 | Number of articles, sentences, tokens and entities (locations, miscellaneous, organizations, and persons) in German data files.

The CoNLL dataset is a widely used resource in the field of natural language processing (NLP). The term “CoNLL” stands for Conference on Natural Language Learning. It originates from a series of shared tasks organized at the Conferences of Natural Language Learning.

Consists of a dataset with 1000 whole scanned receipt images and annotations for the competition on scanned receipts OCR and key information extraction (SROIE). Image source: https://arxiv.org/pdf/2103.10213.pdf

WikiANN, also known as PAN-X, is a multilingual named entity recognition dataset. It consists of Wikipedia articles that have been annotated with LOC (location), PER (person), and ORG (organization) tags in the IOB2 format¹². This dataset serves as a valuable resource for training and evaluating named entity recognition models across various languages. For instance, it includes information about notable individuals, places, and organizations mentioned in Wikipedia articles. Researchers and practitioners can use WikiANN to develop and improve natural language processing systems that identify and classify named entities in text. (1) wikiann · Datasets at Hugging Face. https://huggingface.co/datasets/wikiann. (2) wikiann | TensorFlow Datasets. https://tensorflow.google.cn/datasets/catalog/wikiann. (3) wikiann · Datasets at Hugging Face. https://huggingface.co/datasets/wikiann/viewer/en. (4) WikiAnn Dataset | Papers With Code. https://paperswithcode.com/dataset/wikiann-1.

The shared task of CoNLL-2002 concerns language-independent named entity recognition. The types of named entities include: persons, locations, organizations and names of miscellaneous entities that do not belong to the previous three groups. The participants of the shared task were offered training and test data for at least two languages. Information sources other than the training data might have been used in this shared task. Source: CoNLL 2002 Image Source: https://www.aclweb.org/anthology/W02-2024.pdf

The Cross-lingual TRansfer Evaluation of Multilingual Encoders (XTREME) benchmark was introduced to encourage more research on multilingual transfer learning,. XTREME covers 40 typologically diverse languages spanning 12 language families and includes 9 tasks that require reasoning about different levels of syntax or semantics. The languages in XTREME are selected to maximize language diversity, coverage in existing tasks, and availability of training data. The languages in XTREME are selected to maximize language diversity, coverage in existing tasks, and availability of training data. Among these are many under-studied languages, such as the Dravidian languages Tamil (spoken in southern India, Sri Lanka, and Singapore), Telugu and Malayalam (spoken mainly in southern India), and the Niger-Congo languages Swahili and Yoruba, spoken in Africa.

The IndoNLU benchmark is a collection of resources for training, evaluating, and analyzing natural language understanding systems for Bahasa Indonesia. It is a joint venture from many Indonesia NLP enthusiasts from different institutions such as Gojek, Institut Teknologi Bandung, HKUST, Universitas Multimedia Nusantara, Prosa.ai, and Universitas Indonesia. Source: IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding

Dataset of primarily English Reddit entries which addresses several limitations of prior work. It (1) contains six conceptually distinct primary categories as well as secondary categories, (2) has labels annotated in the context of the conversation thread, (3) contains rationales and (4) uses an expert-driven group-adjudication process for high quality annotations.

WikiNEuRal is a high-quality automatically-generated dataset for Multilingual Named Entity Recognition.

LeNER-Br is a dataset for named entity recognition (NER) in Brazilian Legal Text.

A blackout poetry dataset constructed from publicly available short stories and large poems. The dataset consists of two variants: 8K and 16K examples of passages along with a poem generated from the passage and the indices of the words in the passage from which words in the poem have been selected. The dataset also contains perplexity scores for each of the poems indicating the language quality of the poems. The dataset was constructed synthetically, and hence contains multiple poor poems and frequent grammatical errors. However, it is a great starting point for the task of applying machine learning to blackout poetry generation.

Dataset Description - Authors: Ludovic Moncla, Katherine McDonough and Denis Vigier in the framework of the GEODE project. - Data source: ARTFL Encyclopédie Project, University of Chicago - Github repository: https://github.com/GEODE-project/ner-spancat-edda - Language: French - License: cc-by-nc-4.0 - Zenodo repository: https://zenodo.org/records/10530177 ### Dataset Summary This dataset contains semantic annotations (at the token and span levels) for named entities (such as Spatial, Person, and MISC), nominal entities, as well as nested named entities, spatial relations, and other relevant information within French encyclopedic entries. The span tagset is as follows: - NC-Spatial: a common noun that identifies a spatial entity (nominal spatial entity) including natural features, e.g. ville, la rivière, royaume. - NP-Spatial: a proper noun identifying the name of a place (spatial named entities), e.g. France, Paris, la Chine. - ENE-Spatial: nested spatial entity , e.g. ville de France, royaume de Naples, la mer Baltique. - Relation: spatial relation, e.g. dans, sur, à 10 lieues de. - Latlong: geographic coordinates, e.g. Long. 19. 49. lat. 43. 55. 44. - NC-Person: a common noun that identifies a person (nominal spatial entity), e.g. roi, l'empereur, les auteurs. - NP-Person: a proper noun identifying the name of a person (person named entities), e.g. Louis XIV, Pline, les Romains. - ENE-Person: nested people entity, e.g. le czar Pierre, roi de Macédoine - NP-Misc: a proper noun identifying entities not classified as spatial or person, e.g. l'Eglise, 1702, Pélasgique. - ENE-Misc: nested named entity not classified as spatial or person, e.g. l'ordre de S. Jacques, la déclaration du 21 Mars 1671. - Head: entry name - Domain-Mark: words indicating the knowledge domain (usually after the head and between parenthesis), e.g. Géographie, Geog., en Anatomie. ### Supported Tasks - token-classification or span-classification: The dataset can be used to train a model for token-classification or span-classification. It is more specifically designed for spatial role labelling. A spacy custom spancat model is available at: https://huggingface.co/GEODE/fr_spacy_custom_spancat_edda. ## Dataset Structure The dataset is provided as JSONL files[^1] where each row follows the following structure: { "text": "ILLESCAS, (Géog.) petite ville d'Espagne <...> ", "meta": {"volume": 8, "head": "ILLESCAS", "author": "unsigned", "domain_article": "Géographie", "domain_paragraph": "Géographie", "article": 2637, "paragraph": 1}, "tokens": [{"text": "ILLESCAS", "start": 0, "end": 8, "id": 0, "ws": false}, {"text": ",", "start": 8, "end": 9, "id": 1, "ws": true}, {"text": "(", "start": 10, "end": 11, "id": 2, "ws": false}, {"text": "Géog", "start": 11, "end": 15, "id": 3, "ws": false}, {"text": ".", "start": 15, "end": 16, "id": 4, "ws": false}, {"text": ")", "start": 16, "end": 17, "id": 5, "ws": true}, {"text": "petite", "start": 18, "end": 24, "id": 6, "ws": true}, {"text": "ville", "start": 25, "end": 30, "id": 7, "ws": true}, {"text": "d'", "start": 31, "end": 33, "id": 8, "ws": false}, {"text": "Espagne", "start": 33, "end": 40, "id": 9, "ws": false}, {"text": ",", "start": 40, "end": 41, "id": 10, "ws": true} <...>], "spans": [{"text": "ILLESCAS", "start": 0, "end": 8, "token_start": 0, "token_end": 0, "label": "Head"}, {"text": "Géog.", "start": 11, "end": 16, "token_start": 3, "token_end": 4, "label": "Domain-mark"}, {"text": "petite ville", "start": 18, "end": 30, "token_start": 6, "token_end": 7, "label": "NC-Spatial"}, {"text": "petite ville d'Espagne", "start": 18, "end": 40, "token_start": 6, "token_end": 9, "label": "ENE-Spatial"}, {"text": "petite ville d'Espagne, dans la nouvelle Castille", "start": 18, "end": 67, "token_start": 6, "token_end": 14, "label": "ENE-Spatial"}, {"text": "Espagne", "start": 33, "end": 40, "token_start": 9, "token_end": 9, "label": "NP-Spatial"}, <...>] } Each data contains four main fields: - text: plain text of a paragraph. - meta: metadata from the ARTFL Encyclopédie about the paragraph such volume, article, paragraph id, headword, etc. - tokens: list of tokens, with their text, id, start and end position at the character level. - spans: list of spans (i.e., annotations), with their text, label, start and end position at the character level. [^1]:spaCy binary files are also available on the Github and Zenodo repositories. ### Data Splits The dataset consists of 2200 paragraphs randomly selected out of 2001 Encyclopédie's entries. All paragraphs were written in French and are distributed as follows among the Encyclopédie knowledge domains: | Knowledge domain | Paragraphs | |---|:---:| | Géographie | 1096 | | Histoire | 259 | | Droit Jurisprudence | 113 | | Physique | 92 | | Métiers | 92 | | Médecine | 88 | | Philosophie | 69 | | Histoire naturelle | 65 | | Belles-lettres | 65 | | Militaire | 62 | | Commerce | 48 | | Beaux-arts | 44 | | Agriculture | 36 | | Chasse | 31 | | Religion | 23 | | Musique | 17 | The spans/entities were labeled by the project team along with using pre-labelling with early models to speed up the labelling process. A train/val/test split was used. Validation and test sets are composed of 200 paragraphs each: 100 classified as 'Géographie' and 100 from another knowledge domain. The datasets have the following breakdown of tokens and spans/entities. | | Train | Validation | Test| |---|:---:|:---:|:---:| |Paragraphs| 1,800 | 200 | 200| | Tokens | 132,398 | 14,959 | 13,881 | | NC-Spatial | 3,252 | 358 | 355 | | NP-Spatial | 4,707 | 464 | 519 | | ENE-Spatial | 3,043 | 326 | 334 | | Relation | 2,093 | 219 | 226 | | Latlong | 553 | 66 | 72 | | NC-Person | 1,378 | 132 | 133 | | NP-Person | 1,599 | 170 | 150 | | ENE-Person | 492 | 49 | 57 | | NP-Misc | 948 | 108 | 96 | | ENE-Misc | 255 | 31 | 22 | | Head | 1,261 | 142 | 153 | | Domain-Mark | 1,069 | 122 | 133 | ## Additional Information ### Dataset Curators List of people involved in annotating the dataset: * Ludovic Moncla (@ludovicmoncla), INSA Lyon, CNRS, LIRIS UMR 5205 * Katherine McDonough (@kmcdono2, Lancaster University & The Alan Turing Institute ## Cite this work > Moncla, L., Vigier, D., & McDonough, K. (2024). GeoEDdA: A Gold Standard Dataset for Geo-semantic Annotation of Diderot & d’Alembert’s Encyclopédie. In proceedings of the 2nd International Workshop on Geographic Information Extraction from Texts (GeoExT'24), ECIR Conference, Glasgow, UK. ### Acknowledgement The authors are grateful to the ASLAN project (ANR-10-LABX-0081) of the Université de Lyon, for its financial support within the French program "Investments for the Future" operated by the National Research Agency (ANR). Data courtesy the ARTFL Encyclopédie Project, University of Chicago.

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

The MIM-GOLD-NER dataset is an Icelandic named entity (NE) corpus. It is a version of the MIM-GOLD corpus that has been specifically tagged for named entities. In this dataset, over 48,000 NEs (named entities) are labeled within a corpus of one million tokens. Researchers and developers can use this dataset to train named entity recognizers for Icelandic¹²³. (1) vesteinn/icelandic-ner-MIM-GOLD-NER · Datasets at Hugging Face. https://huggingface.co/datasets/vesteinn/icelandic-ner-MIM-GOLD-NER. (2) svanhvit/icelandic-ner-MIM-GOLD-NER · Datasets at Hugging Face. https://huggingface.co/datasets/svanhvit/icelandic-ner-MIM-GOLD-NER. (3) vesteinn/icelandic-ner-MIM-GOLD-NER at main - Hugging Face. https://huggingface.co/datasets/vesteinn/icelandic-ner-MIM-GOLD-NER/blob/main/icelandic-ner-MIM-GOLD-NER.py. (4) undefined. http://hdl.handle.net/20.500.12537/42.

NERGRIT involves machine learning based NLP Tools and a corpus used for Indonesian Named Entity Recognition, Statement Extraction, and Sentiment Analysis. Source: NERGRIT Corpus

The Tagalog Universal Dependencies NewsCrawl dataset consists of annotated text extracted from the Leipzig Tagalog Corpus. Data included in the Leipzig Tagalog Corpus were crawled from Tagalog-language online news sites by the Leipzig University Institute for Computer Science. The text data was automatically parsed and annotated by Angelina Aquino (University of the Philippines), and then manually corrected according the UD guidelines adapted for Tagalog by Elsie Marie Or (University of the Philippines), Maria Bardají Farré (University of Cologne), and Dr. Nikolaus Himmelmann (University of Cologne). Further verification and automated corrections were done by Lester James Miranda (Allen AI). Due to the source of the data, several typos, grammatical errors, incomplete sentences, and Tagalog-English code-mixing can be found in the dataset. Treebank structure - Train: 12495 sents, 286891 tokens - Dev: 1561 sents, 37045 tokens - Test: 1563 sents, 36974 tokens ###Acknowledgments Aside from the named persons in the previous section, the following also contributed to the project as manual annotators of the dataset: - Patricia Anne Asuncion - Paola Ellaine Luzon - Jenard Tricano - Mary Dianne Jamindang - Michael Wilson Rosero - Jim Bagano - Yeddah Joy Piedad - Farah Cunanan - Calen Manzano - Aien Gengania - Prince Heinreich Omang - Noah Cruz - Leila Ysabelle Suarez - Orlyn Joyce Esquivel - Andre Magpantay The annotation project was made possible by the Deutsche Forschungsgemeinschaft (DFG)-funded project titled "Information distribution and language structure - correlation of grammatical expressions of the noun/verb distinction and lexical information content in Tagalog, Indonesian and German." The DFG project team is composed of Dr. Nikolaus Himmelmann and Maria Bardají Farré from the University of Cologne, and Dr. Gerhard Heyer, Dr. Michael Richter, and Tariq Yousef from the Leipzig University.