Image Quality Assessment

The CSIQ database consists of 30 original images, each is distorted using six different types of distortions at four to five different levels of distortion. CSIQ images are subjectively rated base on a linear displacement of the images across four calibrated LCD monitors placed side by side with equal viewing distance to the observer. The database contains 5000 subjective ratings from 35 different observers, and ratings are reported in the form of DMOS.

AVA is a project that provides audiovisual annotations of video for improving our understanding of human activity. Each of the video clips has been exhaustively annotated by human annotators, and together they represent a rich variety of scenes, recording conditions, and expressions of human activity. There are annotations for: - Kinetics (AVA-Kinetics) - a crossover between AVA and Kinetics. In order to provide localized action labels on a wider variety of visual scenes, authors provide AVA action labels on videos from Kinetics-700, nearly doubling the number of total annotations, and increasing the number of unique videos by over 500x. - Actions (AvA Actions) - the AVA dataset densely annotates 80 atomic visual actions in 430 15-minute movie clips, where actions are localized in space and time, resulting in 1.62M action labels with multiple labels per human occurring frequently. - Spoken Activity (AVA ActiveSpeaker, AVA Speech). AVA ActiveSpeaker: associates speaking activity with a visible face, on the AVA v1.0 videos, resulting in 3.65 million frames labeled across ~39K face tracks. AVA Speech densely annotates audio-based speech activity in AVA v1.0 videos, and explicitly labels 3 background noise conditions, resulting in ~46K labeled segments spanning 45 hours of data. Image Source: https://www.researchgate.net/profile/Paolo_Napoletano/publication/309327222/figure/fig1/AS:419620126248965@1477056642346/Sample-images-from-the-Aesthetic-Visual-Analysis-AVA-database-sorted-by-their-aesthetic.png

KonIQ-10k is a large-scale IQA dataset consisting of 10,073 quality scored images. This is the first in-the-wild database aiming for ecological validity, with regard to the authenticity of distortions, the diversity of content, and quality-related indicators. Through the use of crowdsourcing, we obtained 1.2 million reliable quality ratings from 1,459 crowd workers, paving the way for more general IQA models.

The Smartphone Photography Attribute and Quality (SPAQ) dataset is a comprehensive database for the perceptual quality assessment of smartphone photography. It was introduced in a paper titled "Perceptual Quality Assessment of Smartphone Photography" presented at the IEEE Conference on Computer Vision and Pattern Recognition in 2020. The SPAQ dataset consists of 11,125 pictures taken by 66 smartphones. Each image in the dataset is attached with rich annotations, including: - Image quality - Image attributes (brightness, colorfulness, contrast, noisiness, and sharpness) - Scene category labels (animal, cityscape, human, indoor scene, landscape, night scene, plant, still life, and others) - Exchangeable image file format (EXIF) data These annotations were collected in a well-controlled laboratory environment. The dataset has been used to train blind image quality assessment (BIQA) models, providing insights into how EXIF data, image attributes, and high-level semantics interact with image quality. The SPAQ dataset and the proposed BIQA models are available on GitHub.

Contains aesthetic scores and meaningful attributes assigned to each image by multiple human raters. Source: Photo Aesthetics Ranking Network with Attributes and Content Adaptation

Konstanz artificially distorted image quality database (KADID-10k) contains 81 pristine images, each degraded by 25 distortions in 5 levels.

Dataset with 28,792 retinal images from the EyePACS dataset, based on a three-level quality grading system (i.e., Good', Usable' and `Reject') for evaluating RIQA methods. Source: Evaluation of Retinal Image Quality Assessment Networks in Different Color-spaces

Our dataset was made of videos from MSU Video Upscalers Benchmark Dataset, MSU Video Super-Resolution Benchmark Dataset and MSU Super-Resolution for Video Compression Benchmark Dataset. Dataset consists of real videos (were filmed with 2 cameras), video games footages, movies, cartoons, dynamic ads. How we brought our dataset closer to completeness? * The dataset covers a large number of use cases in the field of SR due to the large number of content types * The dataset contains videos with completely different resolutions, FPS values: 8, 24, 25, 30, 60, as well as high and low spatio-temporal complexity * Distorted videos were obtained using 46 SR methods, some of them were preprocessed with 5 codecs: aomenc, vvenc, x264, x265, uavs3es with different bitrates and qp values * The dataset was manually checked for redundancy Videos from benchmarks are FullHD video crops, since the subjective comparison was made on crops. Therefore, the resolution of all videos in the received dataset is low.

The dataset was created for video quality assessment problem. It was formed with 36 clips from Vimeo, which were selected from 18,000+ open-source clips with high bitrate (license CCBY or CC0). The clips include videos recorded by both professionals and amateurs. Almost half of the videos contain scene changes and high dynamism. Moreover, the synthetic to natural lightning ratio is approximately 1 to 3. * Content type: nature, sport, humans close up, gameplays, music videos, water stream or steam, CGI * Effects and distortions: shaking, slow-motion, grain/noisy, too dark/bright regions, macro shooting, captions (text), extraneous objects on the camera lens or just close to it * Resolution: 1920x1080 as the most popular modern video resolution (more in the future) * Format: yuv420p * FPS: 24, 25, 30, 39, 50, 60 * Videos duration: mainly 10 seconds Such content diversity helps simulate near-realistic conditions. The choice of videos collected for the benchmark dataset employed clustering in terms of space-time complexity to obtain a representative distribution. For compression we used 40 codecs of 10 compression standards (H.264, AV1, H.265, VVC, etc.). Each video was compressed with 3 target bitrates: 1,000 Kbps, 2,000 Kbps, and 4,000 Kbps, and different real-life encoding modes: constant quality (CRF) and variable bitrate (VBR). The choice of bitrate range simplifies the subjective comparison procedure since the video quality is more difficult to distinguish visually at higher bitrates. The subjective assessment involved pairwise comparisons using crowdsourcing service Subjectify.us. To increase the relevance of the results, each pair of videos received at least 10 responses from participants. In total, 766362 valid answers were collected from more than 10800 unique participants.

The dataset was created for video quality assessment problem. It was formed with 36 clips from Vimeo, which were selected from 18,000+ open-source clips with high bitrate (license CCBY or CC0). The clips include videos recorded by both professionals and amateurs. Almost half of the videos contain scene changes and high dynamism. Moreover, the synthetic to natural lightning ratio is approximately 1 to 3. Content type: nature, sport, humans close up, gameplays, music videos, water stream or steam, CGI Effects and distortions: shaking, slow-motion, grain/noisy, too dark/bright regions, macro shooting, captions (text), extraneous objects on the camera lens or just close to it Resolution: 1920x1080 as the most popular modern video resolution (more in the future) Format: yuv420p FPS: 24, 25, 30, 39, 50, 60 Videos duration: mainly 10 seconds Such content diversity helps simulate near-realistic conditions. The choice of videos collected for the benchmark dataset employed clustering in terms of space-time complexity to obtain a representative distribution. For compression we used 40 codecs of 10 compression standards (H.264, AV1, H.265, VVC, etc.). Each video was compressed with 3 target bitrates: 1,000 Kbps, 2,000 Kbps, and 4,000 Kbps, and different real-life encoding modes: constant quality (CRF) and variable bitrate (VBR). The choice of bitrate range simplifies the subjective comparison procedure since the video quality is more difficult to distinguish visually at higher bitrates. The subjective assessment involved pairwise comparisons using crowdsourcing service Subjectify.us. To increase the relevance of the results, each pair of videos received at least 10 responses from participants. In total, 766362 valid answers were collected from more than 10800 unique participants.

TID2013 is a dataset for image quality assessment that contains 25 reference images and 3000 distorted images (25 reference images x 24 types of distortions x 5 levels of distortions). Source: Lukin et al

Includes more than two million traffic sign images that are based on real-world and simulator data. Source: CURE-TSR: Challenging Unreal and Real Environments for Traffic Sign Recognition

Aesthetic Visual Analysis is a dataset for aesthetic image assessment that contains over 250,000 images along with a rich variety of meta-data including a large number of aesthetic scores for each image, semantic labels for over 60 categories as well as labels related to photographic style.

We introduce a novel Image Quality Assessment (IQA) dataset comprising 6073 UHD-1 (4K) images, annotated at a fixed width of 3840 pixels. Contrary to existing No-Reference (NR) IQA datasets, ours focuses on highly aesthetic photos of high technical quality, filling a gap in the literature. The images, carefully curated to exclude synthetic content, are sufficiently diverse to train general NR-IQA models. Importantly, the dataset is annotated with perceptual quality ratings obtained through a crowdsourcing study. Ten expert raters, comprising photographers and graphics artists, assessed each image at least twice in multiple sessions spanning several days, resulting in highly reliable labels. Annotators were rigorously selected based on several metrics, including self-consistency, to ensure their reliability. The dataset includes rich metadata with user and machine-generated tags from over 5,000 categories and popularity indicators such as favorites, likes, downloads, and views. With its unique characteristics, such as its focus on high-quality images, reliable crowdsourced annotations, and high annotation resolution, our dataset opens up new opportunities for advancing perceptual image quality assessment research and developing practical NRIQA models that apply to modern photos.

Briefly describe the dataset. Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset If the description or image is from a different paper, please refer to it as follows: Source: title Image Source: title

The image aesthetic benchmark [18] consists of 10800 Flickr photos of four categories, i.e., “animals”, “urban”, “people” and “nature”, and is constructed originally to retrieve beautiful yet unpopular images in social networks. The ground truths of the photos in the benchmark are five aesthetic grades: “Unacceptable” - images with extremely low quality, out of focus or underexposed, “Flawed” - images with some technical flaws and without any artistic value, “Ordinary” - standard quality images without technical flaws, “Professional” - professional-quality images with some artistic value, and “Exceptional” - very appealing images showing both outstanding professional quality and high artistic value. Image Source: OrdinalCLIP

Year after year, the demand for ever-better smartphone photos continues to grow, in particular in the domain of portrait photography. Manufacturers thus use perceptual quality criteria throughout the development of smartphone cameras. This costly procedure can be partially replaced by automated learning-based methods for image quality assessment (IQA). Due to its subjective nature, it is necessary to estimate and guarantee the consistency of the IQA process, a characteristic lacking in the mean opinion scores (MOS) widely used for crowdsourcing IQA. In addition, existing blind IQA (BIQA) datasets pay little attention to the difficulty of cross-content assessment, which may degrade the quality of annotations. This paper introduces PIQ23, a portrait-specific IQA dataset of 5116 images of 50 predefined scenarios acquired by 100 smartphones, covering a high variety of brands, models, and use cases. The dataset includes individuals of various genders and ethnicities who have given explicit and informed consent for their photographs to be used in public research. It is annotated by pairwise comparisons (PWC) collected from over 30 image quality experts for three image attributes: face detail preservation, face target exposure, and overall image quality. An in-depth statistical analysis of these annotations allows us to evaluate their consistency over PIQ23. Finally, we show through an extensive comparison with existing baselines that semantic information (image context) can be used to improve IQA predictions. The dataset along with the proposed statistical analysis and BIQA algorithms are available: https://github.com/DXOMARKResearch/PIQ2023

To the best of our knowledge, there is no prior dataset specifically constructed for composition assessment. To support the research on this task, we build a dataset upon the existing AADB dataset, from which we collect a total of 9,958 real-world photos. We adopt a composition rating scale from 1 to 5, where a larger score indicates better composition. We make annotation guidelines for composition quality rating and train five individual raters who specialize in fine art. So for each image, we can obtain five composition scores ranging from 1 to 5. Given the subjective nature of human aesthetic activity, we perform sanity check and consistency analysis. We use 240 additional “sanity check” images during annotating to roughly verify the validness of our annotations. We also examine the consistency of composition ratings provided by five individual raters (see Supplementary). We average the composition scores as the ground-truth composition mean score for each image. More details about our CADB dataset will be elaborated in Supplementary. Besides, we observe the content bias in our CADB dataset, that is, there are some biased categories whose score distributions are concentrated in a very narrow interval. After removing 461 biased images, we split the remaining images into 8,547 training images and 950 test images, in which the test set is made less biased for better evaluation (see Supplementary). Cited from "Image Composition Assessment with Saliency-augmented Multi-pattern Pooling" Zhang, Bo and Niu, Li and Zhang, Liqing ## Citation @article{zhang2021image, title={Image Composition Assessment with Saliency-augmented Multi-pattern Pooling}, author={Zhang, Bo and Niu, Li and Zhang, Liqing}, journal={arXiv preprint arXiv:2104.03133}, year={2021} }

Hephaestus is the first large-scale InSAR dataset. Driven by volcanic unrest detection, it provides 19,919 unique satellite frames annotated with a diverse set of labels. Moreover, each sample is accompanied by a textual description of its contents. The goal of this dataset is to boost research on exploitation of interferometric data enabling the application of state-of-the-art computer vision+NLP methods. Furthermore, the annotated dataset is bundled with a large archive of unlabeled frames to enable large-scale self-supervised learning. The final size of the dataset amounts to 110,573 interferograms.

Cross-Reference Omnidirectional Stitching IQA is a novel omnidirectional image dataset containing stitched images as well as dual-fisheye images captured from standard quarters of 0◦, 90◦ , 180◦ and 270◦. In this manner, when evaluating the quality of an image stitched from a pair of fisheye images (e.g., 0◦ and 180◦), the other pair of fisheye images (e.g., 90◦ and 270◦) can be used as the cross-reference to provide ground-truth observations of the stitching regions. Source: Image Quality Assessment for Omnidirectional Cross-reference Stitching

The Fraunhofer Portugal AICOS EDoF Dataset was produced within the TAMI project and is composed of images of microscopic fields of view (FOV) of Liquid-based Cervical Cytology (LBC) samples. A total of 15 LBC samples were supplied by the Pathology Services from Hospital Fernando Fonseca and the Portuguese Oncology Institute of Porto. For each LBC sample, a set of images were obtained using a version of µSmartScope [1,2] prototype adapted to the cervical cytology use case [3,4]. µSmartScope is a portable 3D-printed prototype based on a smartphone developed by Fraunhofer AICOS that allows the fully automatic acquisition of microscopic images. The acquisition is done using the automated focus approach described in [1], where for each FOV, all images in the precise phase of the approach are stored as well as an indication of the image with the best focus metric (standard deviation of the Tenenbaum gradient [5]). The dataset is divided into two partitions: the first partition contains the raw data with 5 images per stack without any pre-processing; the second partition contains the pre-processed data according to the best workflow proposed in [6], and also the respective EDoF generated using Complex Wavelets method for the fusion of the microscopy Images. The images from the second partition were pre-processed using chromatic, static, and elastic alignment; they can be used to fully replicate the work in [6]. The size of every image present in this database is 960x720 pixels. For this work, 144 EDoFs images generated with 5 aligned images per stack are used (where the central image is the one with the best focus metric (C). Figure 1 shows an example of a stack and the respective EDoF. If you find this dataset useful for your research, please cite as: T. Albuquerque, L. Rosado, R. Cruz, M. J. M. Vasconcelos, T. Oliveira J. S. Cardoso, Rethinking Low-Cost Microscopy Workflow: Image Enhancement using Deep Based Extended Depth of Field Methods, In Intelligent Systems with Applications (Elsevier), 2023. https://doi.org/10.1016/j.iswa.2022.200170.

The Reddit Photo Critique Dataset (RPCD) contains tuples of image and photo critiques. RPCD consists of 74K images and 220K comments and is collected from a Reddit community used by hobbyists and professional photographers to improve their photography skills by leveraging constructive community feedback. The proposed dataset differs from previous aesthetics datasets mainly in three aspects, namely * the large scale of the dataset and the extension of the comments criticizing different aspects of the image; * it contains mostly UltraHD images; * it can easily be extended to new data as it is collected through an automatic pipeline.

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset