Optical Character Recognition (OCR)

The COCO (Common Objects in Context) dataset is a large-scale object detection, segmentation, and captioning dataset. It is designed to encourage research on a wide variety of object categories and is commonly used for benchmarking computer vision models. It is an essential dataset for researchers and developers working on object detection, segmentation, and pose estimation tasks.

The MNIST database (Modified National Institute of Standards and Technology database) is a large collection of handwritten digits. It has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger NIST Special Database 3 (digits written by employees of the United States Census Bureau) and Special Database 1 (digits written by high school students) which contain monochrome images of handwritten digits. The digits have been size-normalized and centered in a fixed-size image. The original black and white (bilevel) images from NIST were size normalized to fit in a 20x20 pixel box while preserving their aspect ratio. The resulting images contain grey levels as a result of the anti-aliasing technique used by the normalization algorithm. the images were centered in a 28x28 image by computing the center of mass of the pixels, and translating the image so as to position this point at the center of the 28x28 field. Source: http://yann.lecun.com/exdb/mnist/ Image Source: https://en.wikipedia.org/wiki/MNIST_database#/media/File:MnistExamples.png

The ICDAR 2013 dataset consists of 229 training images and 233 testing images, with word-level annotations provided. It is the standard benchmark dataset for evaluating near-horizontal text detection. Source: Single Shot Text Detector with Regional Attention Image Source: https://plos.figshare.com/articles/Detection_examples_of_the_proposed_method_on_the_ICDAR_2013_dataset_17_/5325856

The IAM database contains 13,353 images of handwritten lines of text created by 657 writers. The texts those writers transcribed are from the Lancaster-Oslo/Bergen Corpus of British English. It includes contributions from 657 writers making a total of 1,539 handwritten pages comprising of 115,320 words and is categorized as part of modern collection. The database is labeled at the sentence, line, and word levels. Source: Measuring Human Perception to Improve Handwritten Document Transcription Image Source: https://fki.tic.heia-fr.ch/databases/iam-handwriting-database

Form Understanding in Noisy Scanned Documents (FUNSD) comprises 199 real, fully annotated, scanned forms. The documents are noisy and vary widely in appearance, making form understanding (FoUn) a challenging task. The proposed dataset can be used for various tasks, including text detection, optical character recognition, spatial layout analysis, and entity labeling/linking. Source: FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents Image source: https://guillaumejaume.github.io/FUNSD/

Contains 145k captions for 28k images. The dataset challenges a model to recognize text, relate it to its visual context, and decide what part of the text to copy or paraphrase, requiring spatial, semantic, and visual reasoning between multiple text tokens and visual entities, such as objects. Source: TextCaps: a Dataset for Image Captioning with Reading Comprehension

ST-VQA aims to highlight the importance of exploiting high-level semantic information present in images as textual cues in the VQA process. Source: Scene Text Visual Question Answering

The ICDAR2003 dataset is a dataset for scene text recognition. It contains 507 natural scene images (including 258 training images and 249 test images) in total. The images are annotated at character level. Characters and words can be cropped from the images. Source: Robust Scene Text Recognition Using Sparse Coding based Features Image Source: https://www.researchgate.net/figure/The-results-of-text-localization-and-extraction-on-ICDAR-2003-dataset_fig3_290070044

TextOCR is a dataset to benchmark text recognition on arbitrary shaped scene-text. TextOCR requires models to perform text-recognition on arbitrary shaped scene-text present on natural images. TextOCR provides ~1M high quality word annotations on TextVQA images allowing application of end-to-end reasoning on downstream tasks such as visual question answering or image captioning. Dataset statistics: - 28,134 natural images from TextVQA - 903,069 annotated scene-text words - 32 words per image on average

SciTSR is a large-scale table structure recognition dataset, which contains 15,000 tables in PDF format and their corresponding structure labels obtained from LaTeX source files. Source: https://github.com/Academic-Hammer/SciTSR

A benchmark dataset that contains 500K document pages with fine-grained token-level annotations for document layout analysis. DocBank is constructed using a simple yet effective way with weak supervision from the \LaTeX{} documents available on the arXiv.com. Source: DocBank: A Benchmark Dataset for Document Layout Analysis

MNIST8M is derived from the MNIST dataset by applying random deformations and translations to the dataset. Source: Scalable and Sustainable Deep Learningvia Randomized Hashing

Description: 105,941 Images Natural Scenes OCR Data of 12 Languages. The data covers 12 languages (6 Asian languages, 6 European languages), multiple natural scenes, multiple photographic angles. For annotation, line-level quadrilateral bounding box annotation and transcription for the texts were annotated in the data. The data can be used for tasks such as OCR of multi-language. Data size: 105,941 images, including Asian language family: Japanese 9,997 images, Korean 10,231 images, Indonesian 7,591 images, Malay 5,650 images, Vietnamese 8,822 images, Thai 9,645 images; European language family: French 10,015 images, German 7,213 images, Italian 8,824 images, Portuguese 7,754 images, Russian 10,376 images and Spanish 9,823 images Collecting environment: including shop plaque, stop board, poster, ticket, road sign, comic, cover picture, prompt/reminder, warning, packing instruction, menu, building sign, etc.

The DeepWeeds dataset consists of 17,509 images capturing eight different weed species native to Australia in situ with neighbouring flora. Source: https://github.com/AlexOlsen/DeepWeeds Image Source: https://github.com/AlexOlsen/DeepWeeds

The DIGITS dataset consists of 1797 8×8 grayscale images (1439 for training and 360 for testing) of handwritten digits. Source: Differentially Private Variational Dropout Image Source: https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_digits.html

The Fudan-ShanghaiTech dataset (FDST) is a dataset for video crowd counting. It contains 15K frames with about 394K annotated heads captured from 13 different scenes Source: https://arxiv.org/abs/1907.07911

CASIA-HWDB is a dataset for handwritten Chinese character recognition. It contains 300 files (240 in HWDB1.1 training set and 60 in HWDB1.1 test set). Each file contains about 3000 isolated gray-scale Chinese character images written by one writer, as well as their corresponding labels. Source: Generating Handwritten Chinese Characters using CycleGAN Image Source: http://www.nlpr.ia.ac.cn/databases/handwriting/Touching_Characters_Databases.html

DialoGLUE is a natural language understanding benchmark for task-oriented dialogue designed to encourage dialogue research in representation-based transfer, domain adaptation, and sample-efficient task learning. It consisting of 7 task-oriented dialogue datasets covering 4 distinct natural language understanding tasks. Source: DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue

The Groove MIDI Dataset (GMD) is composed of 13.6 hours of aligned MIDI and (synthesized) audio of human-performed, tempo-aligned expressive drumming. The dataset contains 1,150 MIDI files and over 22,000 measures of drumming.

This dataset includes 4,500 fully annotated images (over 30,000 license plate characters) from 150 vehicles in real-world scenarios where both the vehicle and the camera (inside another vehicle) are moving. The images were acquired with three different cameras and are available in the Portable Network Graphics (PNG) format with a size of 1,920 × 1,080 pixels. The cameras used were: GoPro Hero4 Silver, Huawei P9 Lite, and iPhone 7 Plus. We collected 1,500 images with each camera, divided as follows: - 900 of cars with gray license plates; - 300 of cars with red license plates; - 300 of motorcycles with gray license plates. The dataset is split as follows: 40% for training, 40% for testing and 20% for validation. Every image has the following annotations available in a text file: the camera in which the image was taken, the vehicle’s position and information such as type (car or motorcycle), manufacturer, model and year; the identification and position of the license plate, as well as the position of its characters. Source: A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector

A benchmark which bridges the gap between freely available, documented, and motivated artificial benchmarks and properties of real industrial problems. The resulting industrial benchmark (IB) has been made publicly available to the RL community by publishing its Java and Python code, including an OpenAI Gym wrapper, on Github. Source: A Benchmark Environment Motivated by Industrial Control Problems

A prebuilt dataset for OpenAI's task for image-2-latex system. Includes total of ~100k formulas and images splitted into train, validation and test sets. Formulas were parsed from LaTeX sources provided here: http://www.cs.cornell.edu/projects/kddcup/datasets.html(originally from arXiv) Each image is a PNG image of fixed size. Formula is in black and rest of the image is transparent. For related tools (eg. tokenizer) check out this repository: https://github.com/Miffyli/im2latex-dataset For pre-made evaluation scripts and built im2latex system check this repository: https://github.com/harvardnlp/im2markup Newlines used in formulas_im2latex.lst are UNIX-style newlines (\n). Reading file with other type of newlines results to slightly wrong amount of lines (104563 instead of 103558), and thus breaks the structure used by this dataset. Python 3.x reads files using newlines of the running system by default, and to avoid this file must be opened with newlines="\n" (eg. open("formulas_im2latex.lst", newline="\n")). Source: https://zenodo.org/record/56198#.YJjuCGZKgox Image source: https://arxiv.org/pdf/1609.04938v2.pdf

Multi-Modal Reading (MMR) Benchmark includes 550 annotated question-answer pairs across 11 distinct tasks involving texts, fonts, visual elements, bounding boxes, spatial relations, and grounding, with carefully designed evaluation metrics.

A new dataset of handwritten text with fine-grained annotations at the character level and report results from an initial user evaluation. Source: DeepWriting: Making Digital Ink Editable via Deep Generative Modeling

The RIMES database (Reconnaissance et Indexation de données Manuscrites et de fac similÉS / Recognition and Indexing of handwritten documents and faxes) was created to evaluate automatic systems of recognition and indexing of handwritten letters. Of particular interest are cases such as those sent by postal mail or fax by individuals to companies or administrations. The database was collected by asking volunteers to write handwritten letters in exchange of gift vouchers. Volunteer were given a fictional identity (same sex as the real one) and up to 5 scenarios. Each scenario has been chosen among 9 realistic following themes : change of personal information (address, bank account), information request, opening and closing (customer account), modification of contract or order, complaint (bad service quality…), payment difficulties (asking for a delay, tax exemption…), reminder letter, damage declaration with further circumstances and a destination (administrations or service providers (telephone, power, bank, insurances). The volunteers composed a letter with those pieces of information using their own words. The layout was free and it was only asked to use white paper and to write in a readable way with black ink. The collect was a success with more than 1,300 people who have participated to the RIMES database creation by writing up to 5 mails. The RIMES database thus obtained contains 12,723 pages corresponding to 5605 mails of two to three pages.

This dataset, called RodoSol-ALPR dataset, contains 20,000 images captured by static cameras located at pay tolls owned by the Rodovia do Sol (RodoSol) concessionaire, which operates 67.5 kilometers of a highway (ES-060) in the Brazilian state of Espírito Santo. There are images of different types of vehicles (e.g., cars, motorcycles, buses and trucks), captured during the day and night, from distinct lanes, on clear and rainy days, and the distance from the vehicle to the camera varies slightly. All images have a resolution of 1,280 × 720 pixels. An important feature of the proposed dataset is that it has images of two different license plate (LP) layouts: Brazilian and Mercosur (to maintain consistency with existing works, we refer to “Brazilian” as the standard used in Brazil before the adoption of the Mercosur standard). Every image has the following information available in a text file: the vehicle’s type (car or motorcycle), the LP’s layout (Brazilian or Mercosul), its text (e.g., ABC-1234), and the position (x, y) of each of its four corners. We labeled the corners instead of just the LP bounding box to enable the training of methods that explore LP rectification, as well as the application of a wider range of data augmentation techniques. Regarding privacy concerns related to our dataset, we remark that in Brazil the LPs are related to the respective vehicles, i.e., no public information is available about the vehicle drivers/owners. Moreover, all human faces (e.g., drivers or RodoSol’s employees) were manually redacted (i.e., blurred) in each image.

This dataset aims at evaluating the License Plate Character Segmentation (LPCS) problem. The experimental results of the paper Benchmark for License Plate Character Segmentation were obtained using a dataset providing 101 on-track vehicles captured during the day. The video was recorded using a static camera in early 2015. The images of the dataset were acquired with a digital camera in Full-HD and are available in the Portable Network Graphics (PNG) format with 1920×1080 pixels each. The average size of each file is 4.08 Megabytes (a total of 8.60 Gigabytes for the entire dataset). In addition, since there are some approaches that track the car to utilize redundant information to improve the recognition results, we decided to make a dataset with multiples frames per car. In this dataset, there are, on average, 19.80 image frames per vehicle (with a standard deviation of 4.14). Source: Benchmark for License Plate Character Segmentation

The Kannada-MNIST dataset is a drop-in substitute for the standard MNIST dataset for the Kannada language. Source: https://github.com/vinayprabhu/Kannada_MNIST Image Source: https://github.com/vinayprabhu/Kannada_MNIST

To benchmark Bengali digit recognition algorithms, a large publicly available dataset is required which is free from biases originating from geographical location, gender, and age. With this aim in mind, NumtaDB, a dataset consisting of more than 85,000 images of hand-written Bengali digits, has been assembled.

IIIT-AR-13K is created by manually annotating the bounding boxes of graphical or page objects in publicly available annual reports. This dataset contains a total of 13k annotated page images with objects in five different popular categories - table, figure, natural image, logo, and signature. It is the largest manually annotated dataset for graphical object detection. Source: IIIT-AR-13K: A New Dataset for Graphical Object Detection in Documents Image Source: http://cvit.iiit.ac.in/usodi/iiitar13k.php

Contains video clips shot with modern high-resolution mobile cameras, with strong projective distortions and with low lighting conditions. Source: MIDV-2019: Challenges of the modern mobile-based document OCR

MLe2 is a dataset for the evaluation of scene text end-to-end reading systems and all intermediate stages such as text detection, script identification and text recognition. The dataset contains a total of 711 scene images covering four different scripts (Latin, Chinese, Kannada, and Hangul). Source: A fine-grained approach to scene text script identification

This dataset arises from the READ project (Horizon 2020). The dataset consists of a subset of documents from the Ratsprotokolle collection composed of minutes of the council meetings held from 1470 to 1805 (about 30.000 pages), which will be used in the READ project. This dataset is written in Early Modern German. The number of writers is unknown. Handwriting in this collection is complex enough to challenge the HTR software. The training dataset is composed of 400 pages; most of the pages consist of a single block with many difficulties for line detection and extraction. The ground-truth in this set is in PAGE format and it is provided annotated at line level in the PAGE files. The previous dataset is the same that is located at https://zenodo.org/record/218236#.WnLhaCHhBGF The new file includes the test set corresponding to the HTR competition held at ICFHR 2016 Toselli, A.H., Romero, V., Villegas, M., Vidal, E., & Sánchez, J.A. (2018). HTR Dataset ICFHR 2016 (Version 1.2.0) [Data set]. Zenodo. http://doi.org/10.5281/zenodo.1297399

The BanglaWriting dataset contains single-page handwritings of 260 individuals of different personalities and ages. Each page includes bounding-boxes that bounds each word, along with the unicode representation of the writing. This dataset contains 21,234 words and 32,787 characters in total. Moreover, this dataset includes 5,470 unique words of Bangla vocabulary. Apart from the usual words, the dataset comprises 261 comprehensible overwriting and 450 incomprehensible overwriting. All of the bounding boxes and word labels are manually-generated. The dataset can be used for complex optical character/word recognition, writer identification, and handwritten word segmentation. Furthermore, this dataset is suitable for extracting age-based and gender-based variation of handwriting. Source: https://github.com/QuwsarOhi/BanglaWriting Image Source: https://github.com/QuwsarOhi/BanglaWriting

Chinese Text in the Wild is a dataset of Chinese text with about 1 million Chinese characters from 3850 unique ones annotated by experts in over 30000 street view images. This is a challenging dataset with good diversity containing planar text, raised text, text under poor illumination, distant text, partially occluded text, etc.