Image Enhancement

CoNLL-2003 is a named entity recognition dataset released as a part of CoNLL-2003 shared task: language-independent named entity recognition. The data consists of eight files covering two languages: English and German. For each of the languages there is a training file, a development file, a test file and a large file with unannotated data. The English data was taken from the Reuters Corpus. This corpus consists of Reuters news stories between August 1996 and August 1997. For the training and development set, ten days worth of data were taken from the files representing the end of August 1996. For the test set, the texts were from December 1996. The preprocessed raw data covers the month of September 1996. The text for the German data was taken from the ECI Multilingual Text Corpus. This corpus consists of texts in many languages. The portion of data that was used for this task, was extracted from the German newspaper Frankfurter Rundshau. All three of the training, development and test sets were taken from articles written in one week at the end of August 1992. The raw data were taken from the months of September to December 1992. | English data | Articles | Sentences | Tokens | LOC | MISC | ORG | PER | |-------------------|----------|-----------|---------|------|------|------|------| | Training set | 946 | 14,987 | 203,621 | 7140 | 3438 | 6321 | 6600 | | Development set | 216 | 3,466 | 51,362 | 1837 | 922 | 1341 | 1842 | | Test set | 231 | 3,684 | 46,435 | 1668 | 702 | 1661 | 1617 | Number of articles, sentences, tokens and entities (locations, miscellaneous, organizations, and persons) in English data files. | German data | Articles | Sentences | Tokens | LOC | MISC | ORG | PER | |-------------------|----------|-----------|---------|------|------|------|------| | Training set | 553 | 12,705 | 206,931 | 4363 | 2288 | 2427 | 2773 | | Development set | 201 | 3,068 | 51,444 | 1181 | 1010 | 1241 | 1401 | | Test set | 155 | 3,160 | 51,943 | 1035 | 670 | 773 | 1195 | Number of articles, sentences, tokens and entities (locations, miscellaneous, organizations, and persons) in German data files.

The New York Times Annotated Corpus contains over 1.8 million articles written and published by the New York Times between January 1, 1987 and June 19, 2007 with article metadata provided by the New York Times Newsroom, the New York Times Indexing Service and the online production staff at nytimes.com. The corpus includes: - Over 1.8 million articles (excluding wire services articles that appeared during the covered period). - Over 650,000 article summaries written by library scientists. - Over 1,500,000 articles manually tagged by library scientists with tags drawn from a normalized indexing vocabulary of people, organizations, locations and topic descriptors. - Over 275,000 algorithmically-tagged articles that have been hand verified by the online production staff at nytimes.com. As part of the New York Times' indexing procedures, most articles are manually summarized and tagged by a staff of library scientists. This collection contains over 650,000 article-summary pairs which may prove to be useful in the development and evaluation of algorithms for automated document summarization. Also, over 1.5 million documents have at least one tag. Articles are tagged for persons, places, organizations, titles and topics using a controlled vocabulary that is applied consistently across articles. For instance if one article mentions "Bill Clinton" and another refers to "President William Jefferson Clinton", both articles will be tagged with "CLINTON, BILL". Source: https://catalog.ldc.upenn.edu/LDC2008T19

The LOL dataset is composed of 500 low-light and normal-light image pairs and divided into 485 training pairs and 15 testing pairs. The low-light images contain noise produced during the photo capture process. Most of the images are indoor scenes. All the images have a resolution of 400×600. Source: Unsupervised Real-world Low-light Image Enhancement with Decoupled Networks Image Source: https://daooshee.github.io/BMVC2018website/

OntoNotes 5.0 is a large corpus comprising various genres of text (news, conversational telephone speech, weblogs, usenet newsgroups, broadcast, talk shows) in three languages (English, Chinese, and Arabic) with structural information (syntax and predicate argument structure) and shallow semantics (word sense linked to an ontology and coreference). OntoNotes Release 5.0 contains the content of earlier releases - and adds source data from and/or additional annotations for, newswire, broadcast news, broadcast conversation, telephone conversation and web data in English and Chinese and newswire data in Arabic. Source: https://catalog.ldc.upenn.edu/LDC2013T19

BC5CDR corpus consists of 1500 PubMed articles with 4409 annotated chemicals, 5818 diseases and 3116 chemical-disease interactions. Source: https://www.ncbi.nlm.nih.gov/research/bionlp/Data/ Image Source: https://arxiv.org/pdf/1805.10586.pdf

The 3DMATCH benchmark evaluates how well descriptors (both 2D and 3D) can establish correspondences between RGB-D frames of different views. The dataset contains 2D RGB-D patches and 3D patches (local TDF voxel grid volumes) of wide-baselined correspondences. The pixel size of each 2D patch is determined by the projection of the 0.3m3 local 3D patch around the interest point onto the image plane. Source: 3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions

The See-in-the-Dark (SID) dataset contains 5094 raw short-exposure images, each with a corresponding long-exposure reference image. Images were captured using two cameras: Sony α7SII and Fujifilm X-T2.

The Annotated Facial Landmarks in the Wild (AFLW) is a large-scale collection of annotated face images gathered from Flickr, exhibiting a large variety in appearance (e.g., pose, expression, ethnicity, age, gender) as well as general imaging and environmental conditions. In total about 25K faces are annotated with up to 21 landmarks per image. Source: Nose, Eyes and Ears: Head Pose Estimation by Locating Facial Keypoints

The NCBI Disease corpus consists of 793 PubMed abstracts, which are separated into training (593), development (100) and test (100) subsets. The NCBI Disease corpus is annotated with disease mentions, using concept identifiers from either MeSH or OMIM. Source: A Neural Multi-Task Learning Framework to Jointly Model Medical Named Entity Recognition and Normalization

SciERC dataset is a collection of 500 scientific abstract annotated with scientific entities, their relations, and coreference clusters. The abstracts are taken from 12 AI conference/workshop proceedings in four AI communities, from the Semantic Scholar Corpus. SciERC extends previous datasets in scientific articles SemEval 2017 Task 10 and SemEval 2018 Task 7 by extending entity types, relation types, relation coverage, and adding cross-sentence relations using coreference links. Source: http://nlp.cs.washington.edu/sciIE/ Image Source: http://nlp.cs.washington.edu/sciIE/

This shared task focuses on identifying unusual, previously-unseen entities in the context of emerging discussions. Named entities form the basis of many modern approaches to other tasks (like event clustering and summarisation), but recall on them is a real problem in noisy text - even among annotators. This drop tends to be due to novel entities and surface forms. Take for example the tweet “so.. kktny in 30 mins?” - even human experts find entity kktny hard to detect and resolve. This task will evaluate the ability to detect and classify novel, emerging, singleton named entities in noisy text. The goal of this task is to provide a definition of emerging and of rare entities, and based on that, also datasets for detecting these entities.

The GENIA corpus is the primary collection of biomedical literature compiled and annotated within the scope of the GENIA project. The corpus was created to support the development and evaluation of information extraction and text mining systems for the domain of molecular biology. The corpus contains 1,999 Medline abstracts, selected using a PubMed query for the three MeSH terms “human”, “blood cells”, and “transcription factors”. The corpus has been annotated with various levels of linguistic and semantic information. The primary categories of annotation in the GENIA corpus and the corresponding subcorpora are: * Part-of-Speech annotation * Constituency (phrase structure) syntactic annotation * Term annotation * Event annotation * Relation annotation * Coreference annotation Source: http://www.geniaproject.org/genia-corpus Image Source: http://www.geniaproject.org/genia-corpus

• Visible-infrared Paired Dataset for Low-light Vision * 30976 images (15488 pairs) * 24 dark scenes, 2 daytime scenes * Support for image-to-image translation (visible to infrared, or infrared to visible), visible and infrared image fusion, low-light pedestrian detection, and infrared pedestrian detection * (The original image and video pairs (before registration) of LLVIP are also released!)

Includes 950 real-world underwater images, 890 of which have the corresponding reference images. Source: An Underwater Image Enhancement Benchmark Dataset and Beyond

DICM is a dataset for low-light enhancement which consists of 69 images collected with commercial digital cameras. Source: Deep Retinex Decomposition for Low-Light Enhancement Image Source: GLADNet: Low-Light Enhancement Network with Global Awareness

WikiANN, also known as PAN-X, is a multilingual named entity recognition dataset. It consists of Wikipedia articles that have been annotated with LOC (location), PER (person), and ORG (organization) tags in the IOB2 format¹². This dataset serves as a valuable resource for training and evaluating named entity recognition models across various languages. For instance, it includes information about notable individuals, places, and organizations mentioned in Wikipedia articles. Researchers and practitioners can use WikiANN to develop and improve natural language processing systems that identify and classify named entities in text. (1) wikiann · Datasets at Hugging Face. https://huggingface.co/datasets/wikiann. (2) wikiann | TensorFlow Datasets. https://tensorflow.google.cn/datasets/catalog/wikiann. (3) wikiann · Datasets at Hugging Face. https://huggingface.co/datasets/wikiann/viewer/en. (4) WikiAnn Dataset | Papers With Code. https://paperswithcode.com/dataset/wikiann-1.

The Exclusively Dark (ExDARK) dataset is a collection of 7,363 low-light images from very low-light environments to twilight (i.e 10 different conditions) with 12 object classes (similar to PASCAL VOC) annotated on both image class level and local object bounding boxes. Source: https://github.com/cs-chan/Exclusively-Dark-Image-Dataset

ACE 2004 Multilingual Training Corpus contains the complete set of English, Arabic and Chinese training data for the 2004 Automatic Content Extraction (ACE) technology evaluation. The corpus consists of data of various types annotated for entities and relations and was created by Linguistic Data Consortium with support from the ACE Program, with additional assistance from the DARPA TIDES (Translingual Information Detection, Extraction and Summarization) Program. The objective of the ACE program is to develop automatic content extraction technology to support automatic processing of human language in text form. In September 2004, sites were evaluated on system performance in six areas: Entity Detection and Recognition (EDR), Entity Mention Detection (EMD), EDR Co-reference, Relation Detection and Recognition (RDR), Relation Mention Detection (RMD), and RDR given reference entities. All tasks were evaluated in three languages: English, Chinese and Arabic. Source: https://catalog.ldc.upenn.edu/LDC2005T09

PolyU Dataset is a large dataset of real-world noisy images with reasonably obtained corresponding “ground truth” images. The basic idea is to capture the same and unchanged scene for many (e.g., 500) times and compute their mean image, which can be roughly taken as the “ground truth” image for the real-world noisy images. The rational of this strategy is that for each pixel, the noise is generated randomly larger or smaller than 0. Sampling the same pixel many times and computing the average value will approximate the truth pixel value and alleviate significantly the noise. Source: Real-world Noisy Image Denoising: A New Benchmark

The MIT-Adobe FiveK dataset consists of 5,000 photographs taken with SLR cameras by a set of different photographers. They are all in RAW format; that is, all the information recorded by the camera sensor is preserved. We made sure that these photographs cover a broad range of scenes, subjects, and lighting conditions. We then hired five photography students in an art school to adjust the tone of the photos. Each of them retouched all the 5,000 photos using a software dedicated to photo adjustment (Adobe Lightroom) on which they were extensively trained. We asked the retouchers to achieve visually pleasing renditions, akin to a postcard. The retouchers were compensated for their work. This dataset was collected for our project on learning photographic adjustments. When using images from this dataset, please cite this dataset using the following BibTeX: @inproceedings{fivek, author = "Vladimir Bychkovsky and Sylvain Paris and Eric Chan and Fr{\'e}do Durand", title = "Learning Photographic Global Tonal Adjustment with a Database of Input / Output Image Pairs", booktitle = "The Twenty-Fourth IEEE Conference on Computer Vision and Pattern Recognition", year = "2011" } Source: https://data.csail.mit.edu/graphics/fivek/ Image source: https://data.csail.mit.edu/graphics/fivek/

LOL-v2-real contains 689 low-/normal-light image pairs for training and 100 pairs for testing.

A dataset of images taken in different locations with varying water properties, showing color charts in the scenes. Moreover, to obtain ground truth, the 3D structure of the scene was calculated based on stereo imaging. This dataset enables a quantitative evaluation of restoration algorithms on natural images. Source: Underwater Single Image Color Restoration Using Haze-Lines and a New Quantitative Dataset

Various documents dataset. Each of the 65 documents includes scanned ground truth images, both hard and easy distorted photos, and document-centered cropped images. Image Source: DocUNet: Document Image Unwarping via a Stacked U-Net

This is the low-light image enhancement dataset collected by the CVPR 2018 paper "Seeing Motion in the Dark".

VIDIT is a reference evaluation benchmark and to push forward the development of illumination manipulation methods. VIDIT includes 390 different Unreal Engine scenes, each captured with 40 illumination settings, resulting in 15,600 images. The illumination settings are all the combinations of 5 color temperatures (2500K, 3500K, 4500K, 5500K and 6500K) and 8 light directions (N, NE, E, SE, S, SW, W, NW). Original image resolution is 1024x1024. Source: VIDIT: Virtual Image Dataset for Illumination Transfer Image source: https://github.com/majedelhelou/VIDIT

We introduce Stanford-ORB, a new real-world 3D Object inverse Rendering Benchmark. Recent advances in inverse rendering have enabled a wide range of real-world applications in 3D content generation, moving rapidly from research and commercial use cases to consumer devices. While the results continue to improve, there is no real-world benchmark that can quantitatively assess and compare the performance of various inverse rendering methods. Existing real-world datasets typically only consist of the shape and multi-view images of objects, which are not sufficient for evaluating the quality of material recovery and object relighting. Methods capable of recovering material and lighting often resort to synthetic data for quantitative evaluation, which on the other hand does not guarantee generalization to complex real-world environments. We introduce a new dataset of real-world objects captured under a variety of natural scenes with ground-truth 3D scans, multi-view images, and environment lighting. Using this dataset, we establish the first comprehensive real-world evaluation benchmark for object inverse rendering tasks from in-the-wild scenes, and compare the performance of various existing methods.

A dataset of over 24,000 images exhibiting the broadest range of exposure values to date with a corresponding properly exposed image.

Created by Smith et al. at 2008, the BioCreative II Gene Mention Recognition (BC2GM) Dataset contains data where participants are asked to identify a gene mention in a sentence by giving its start and end characters. The training set consists of a set of sentences, and for each sentence a set of gene mentions (GENE annotations). [registration required for access], in English language. Containing 20 in n/a file format.

The real captured dataset of LOL contains 500 low/normallight image pairs. Most low-light images are collected by changing exposure time and ISO, while other configurations of the cameras are fixed. We capture images from a variety of scenes, e.g., houses, campuses, clubs, streets. Since camera shaking, object movement, and lightness changing may cause misalignment between the image pairs, inspired by [41], a three-step shooting strategy is used to eliminate such misalignments between the image pairs in our dataset. For one scene, we first shoot two normal-light images $N_1$ and $N_2$. Then, we change the exposure time and ISO to capture a series of low-light images. Finally, we set the exposure time and ISO back to shoot another two normal-light images $N_3$ and $N_4$. The average of $N_i (i = 1,2,3,4)$ is treated as the ground-truth $G=\frac{1}{4}\sum^4_{i=1}N_i$. Then, we check whether there is object or camera movement. Specifically, the misalignment for these normal-light images is measured by $M=\frac{1}{4}\sum^4_{i=1}MSE(Ni, G)$. If M > 0.1, we abandon the corresponding pair. These raw images are resized to 400 × 600 and converted to Portable Network Graphics format. The dataset is publicly available.

From Fidelity to Perceptual Quality: A Semi-Supervised Approach for Low-Light Image Enhancement

The first large demoire dataset. The dataset contains 135,000 image pairs, each containing an image contaminated with moire patterns and its corresponding uncontaminated reference image.

To make synthetic images match the property of real dark photography, we analyze the illumination distribution of low-light images. We collect 270 low-light images from public MEF [42], NPE [6], LIME [8], DICM [43], VV,2 and Fusion [44] dataset, transform the imagesT into YCbCr channel and calculate the histogram of Y channel. We also collect 1000 raw images from RAISE [45] as normal-light images and calculate the histogram of Y channel in YCbCr. Raw images contain more information than the converted results. For raw images, all operations used to generate pixel values are performed in one step on the base data, making the result more accurate. 1000 raw images in RAISE [45] are used to synthesize low-light images. Interface provided by Adobe Lightroom is used and we try different kinds of parameters to make the histogram of Y channel fit the result in low-light images. Final parameter configuration can be found in the supplementary material. The illumination distribution of synthetic images matches that of low-light images. Finally, we resize these raw images to 400 × 600 and convert them to Portable Network Graphics format.

Multi-exposure image fusion (MEF) is considered an effective quality enhancement technique widely adopted in consumer electronics, but little work has been dedicated to the perceptual quality assessment of multi-exposure fused images. In this paper, we first build an MEF database and carry out a subjective user study to evaluate the quality of images generated by different MEF algorithms. There are several useful findings. First, considerable agreement has been observed among human subjects on the quality of MEF images. Second, no single state-of-the-art MEF algorithm produces the best quality for all test images. Third, the existing objective quality models for general image fusion are very limited in predicting perceived quality of MEF images. Motivated by the lack of appropriate objective models, we propose a novel objective image quality assessment (IQA) algorithm for MEF images based on the principle of the structural similarity approach and a novel measure of patch structural consistency. Our experimental results on the subjective database show that the proposed model well correlates with subjective judgments and significantly outperforms the existing IQA models for general image fusion. Finally, we demonstrate the potential application of the proposed model by automatically tuning the parameters of MEF algorithms

The dataset is maintained by VISION AND IMAGE PROCESSING LAB, University of Waterloo. The images of the dataset were extracted from the public databases DermIS and DermQuest, along with manual segmentations of the lesions. The dataset was used in the following journal publication. [1] Glaister, J., A. Wong, and D. A. Clausi, "Automatic segmentation of skin lesions from dermatological photographs using a joint probabilistic texture distinctiveness approach", IEEE Transactions on Biomedical Engineering [2] Amelard, R., J. Glaister, A. Wong, and D. A. Clausi, "High-level intuitive features (HLIFs) for intuitive skin lesion descriptionpdf", IEEE Transactions on Biomedical Engineering, vol. 62, issue 3, pp. 820-831, October, 2015. [3] Glaister, J., R. Amelard, A. Wong, and D. A. Clausi, "MSIM: Multi-Stage Illumination Modeling of Dermatological Photographs for Illumination-Corrected Skin Lesion Analysis", IEEE Transactions on Biomedical Engineering, vol. 60, issue 7, pp. 1873 - 1883, November, 2013.

LoLi-Phone is a large-scale low-light image and video dataset for Low-light image enhancement (LLIE). The images and videos are taken by different mobile phones' cameras under diverse illumination conditions.

The dataset collected by the paper Seeing Dynamic Scene in the Dark: High-Quality Video Dataset with Mechatronic Alignment, ICCV 2021