This dataset is a new benchmark, grounded in real-world usages is developed to support more authentic and comprehensive evaluation of image editing models.

The German Lipreading dataset consists of 250,000 publicly available videos of the faces of speakers of the Hessian Parliament, which was processed for word-level lip reading using an automatic pipeline. The format is similar to that of the English language Lip Reading in the Wild (LRW) dataset, with each H264-compressed MPEG-4 video encoding one word of interest in a context of 1.16 seconds duration, which yields compatibility for studying transfer learning between both datasets. Choosing video material based on naturally spoken language in a natural environment ensures more robust results for real-world applications than artificially generated datasets with as little noise as possible. The 500 different spoken words ranging between 4-18 characters in length each have 500 instances and separate MPEG-4 audio- and text metadata-files, originating from 1018 parliamentary sessions. Additionally, the complete TextGrid files containing the segmentation information of those sessions are also included. The size of the uncompressed dataset is 15GB.

HRS-Bench is a concrete evaluation benchmark for T2I models that is Holistic, Reliable, and Scalable. It measures 13 skills that can be categorized into five major categories: accuracy, robustness, generalization, fairness, and bias. In addition, HRS-Bench covers 50 scenarios, including fashion, animals, transportation, food, and clothes. Source: HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models Image Source: HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models

YouTube Driving Dataset contains a massive amount of real-world driving frames with various conditions, from different weather, different regions, to diverse scene types

Contains 8k flickr Images with captions. Visit this page to explore the data. Cite this paper if you find it useful in your research: Framing image description as a ranking task: data, models and evaluation metrics

MatSynth MatSynth is a Physically Based Rendering (PBR) materials dataset designed for modern AI applications. This dataset consists of over 4,000 ultra-high resolution, offering unparalleled scale, diversity, and detail. Meticulously collected and curated, MatSynth is poised to drive innovation in material acquisition and generation applications, providing a rich resource for researchers, developers, and enthusiasts in computer graphics and related fields. ## Dataset Description MatSynth is a new large-scale dataset comprising over 4,000 ultra-high resolution Physically Based Rendering (PBR) materials, all released under permissive licensing. All materials in the dataset are represented by a common set of maps (Basecolor, Diffuse, Normal, Height, Roughness, Metallic, Specular and, when useful, Opacity), modelling both the reflectance and mesostructure of the material. Each material in the dataset comes with rich metadata, including information on its origin, licensing details, category, tags, creation method, and, when available, descriptions and physical size. This comprehensive metadata facilitates precise material selection and usage, catering to the specific needs of users. ## Dataset Structure The MatSynth dataset is divided into two splits: the test split, containing 89 materials, and the train split, consisting of 3,980 materials. To enhance accessibility and ease of navigation, each split is further organized into separate folders for each distinct category present in the dataset (Blends, Ceramic, Concrete, Fabric, Ground, Leather, Marble, Metal, Misc, Plastic, Plaster, Stone, Terracotta, Wood). ## Dataset Creation The MatSynth dataset is designed to support modern, learning-based techniques for a variety of material-related tasks including, but not limited to, material acquisition, material generation and synthetic data generation e.g. for retrieval or segmentation. ## Source Data The MatSynth dataset is the result of an extensively collection of data from multiple online sources operating under the CC0 and CC-BY licensing framework. This collection strategy allows to capture a broad spectrum of materials, from commonly used ones to more niche or specialized variants while guaranteeing that the data can be used for a variety of usecases. Materials under CC0 license were collected from AmbientCG, CGBookCase, PolyHeaven, ShateTexture, and TextureCan. The dataset also includes limited set of materials from the artist Julio Sillet, distributed under CC-BY license. We collected over 6000 materials which we meticulously filter to keep only tileable, 4K materials. This high resolution allows us to extract many different crops from each sample at different scale for augmentation. Additionally, we discard blurry or low-quality materials (by visual inspection). The resulting dataset consists of 3736 unique materials which we augment by blending semantically compatible materials (e.g.: snow over ground). In total, our dataset contains 4069 unique 4K materials. ## Annotations The dataset is composed of material maps (Basecolor, Diffuse, Normal, Height, Roughness, Metallic, Specular and, when useful, opacity) and associated renderings under varying environmental illuminations, and multi-scale crops. We adopt the OpenGL standard for the Normal map (Y-axis pointing upward). The Height map is given in a 16-bit single channel format for higher precision. In addition to these maps, the dataset includes other annotations providing context to each material: the capture method (photogrammetry, procedural generation, or approximation); list of descriptive tags; source name (website); source link; licensing and a timestamps for eventual future versioning. For a subset of materials, when the information is available, we also provide the author name (387), text description (572) and a physical size, presented as the length of the edge in centimeters (358). ## Citation @article{vecchio2024matsynth, title={MatSynth: A Modern PBR Materials Dataset}, author={Vecchio, Giuseppe and Deschaintre, Valentin}, journal={arXiv preprint arXiv:2401.06056}, year={2024} }

The ObjectsRoom dataset is based on the MuJoCo environment used by the Generative Query Network [4] and is a multi-object extension of the 3d-shapes dataset. The training set contains 1M scenes with up to three objects. We also provide ~1K test examples for the following variants: 2.1 Empty room: scenes consist of the sky, walls, and floor only. 2.2 Six objects: exactly 6 objects are visible in each image. 2.3 Identical color: 4-6 objects are placed in the room and have an identical, randomly sampled color. Datapoints consist of an image and fixed number of masks. The first four masks correspond to the sky, floor, and two halves of the wall respectively. The remaining masks correspond to the foreground objects. Source: Objects Room

RC-49 is a benchmark dataset for generating images conditional on a continuous scalar variable. It is made by rendering 49 3-D chair models from ShapeNet individually. Each chair model is rendered at 899 yaw angles from $0.1^{\circ}$ to $89.9^{\circ}$ with a stepsize of $0.1^{\circ}$. This dataset contains 44,051 RGB images of size $64\times64$ with corresponding yaw angles as labels. Note that in CcGAN, angles are used for training if their last digits are odd. Thus, there are 450 angles in the training set. Moreover, for these 450 training angles, only 25 images for each angle are used for the training.

• The Sims4Action Dataset: a videogame-based dataset for Synthetic→Real domain adaptation for human activity recognition. * Goal : Exploring the concept of constructing training examples for Activities of Daily Living (ADL) recognition by playing life simulation video games. * ** Sims4Action dataset is created with the commercial game THE SIMS 4** by executing actions-of-interest within the game in a "top-down" manner. It features ten hours of video material of eight diverse characters and multiple environments. Ten actions are selected to have a direct correspondence to categories covered in the real-life dataset Toyota Smarthome [2] to enable the research of Synthetic→Real transfer in action recognition. * Two benchmarks :* Gaming→Gaming* (training and evaluation on Sims4Action) and Gaming→Real (training on Sims4Action, evaluation on the real Toyota Smarthome data [2]). * Main challenge: Gaming→Real domain adaptation While ADL recognition on gaming data is interesting from a theoretical perspective, the key challenge arises from transferring knowledge learned from simulated data to real-world applications. Sims4Action specifically provides a benchmark for this scenario since it describes a Gaming→Real challenge, which evaluates models on real videos derived from the existing Toyota Smarthome dataset . # References [1] Let's Play for Action: Recognizing Activities of Daily Living by Learning from Life Simulation Video Games. Alina Roitberg*, David Schneider*, Aulia Djamal, Constantin Seibold, Simon Reiß, Rainer Stiefelhagen, In International Conference on Intelligent Robots and Systems (IROS), 2021 (* denotes equal contribution.) [2] Toyota smarthome: Real-world activities of daily living. Srijan Das, Rui Dai, Michal Koperski, Luca Minciullo, Lorenzo Garattoni, Francois Bremond, Gianpiero Francesca, In International Conference on Computer Vision (ICCV), 2019.

StreetStyle is a large-scale dataset of photos of people annotated with clothing attributes, and use this dataset to train attribute classifiers via deep learning. Source: StreetStyle: Exploring world-wide clothing styles from millions of photos Image Source: Matzen et al

A Dense-text Image Benchmark to evaluate large generation model's ability on text generation.

This dataset is an OSN-transmitted (OSN = Online Social Network) version of the CASIA dataset. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (OSN = Online Social Network) version of the CASIA dataset. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (OSN = Online Social Network) version of the CASIA dataset. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the Columbia dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the DSO dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

The eSports Sensors dataset contains sensor data collected from 10 players in 22 matches in League of Legends. The sensor data collected includes: * Hand/head/chair movements. * Heart rate. * Muscle activity. * Gaze movement on the monitor. * Galvanic skin response(GSR). * Electroencephalography (EEG). * Mouse and keyboard activity. * Facial skin temperature. * Environmental data. The data were collected for one team of 5 people simultaneously. In-game logs and meta information for each match are also provided for each match. Source: https://github.com/smerdov/eSports_Sensors_Dataset

The FIGR-8 database is a dataset containing 17,375 classes of 1,548,256 images representing pictograms, ideograms, icons, emoticons or object or conception depictions. Its aim is to set a benchmark for Few-shot Image Generation tasks, albeit not being limited to it. Each image is represented by 192x192 pixels with grayscale value of 0-255. Classes are not balanced (they do not all contain the same number of elements), but they all do contain at the very least 8 images. Source: https://github.com/marcdemers/FIGR-8 Image Source: https://github.com/marcdemers/FIGR-8

The database is written in Cyrillic and shares the same 33 characters. Besides these characters, the Kazakh alphabet also contains 9 additional specific characters. This dataset is a collection of forms. The sources of all the forms in the datasets were generated by LATEX which subsequently was filled out by persons with their handwriting. The database consists of more than 1400 filled forms. There are approximately 63000 sentences, more than 715699 symbols produced by approximately 200 diferent writers. We utilized three different datasets described as following: Handwritten samples (Forms) of keywords in Kazakh and Russian (Areas, Cities , Village , etc.) Handwritten Kazakh and Russian alphabet in cyrillic Handwritten samples (Forms) of poems in Russian Image source: https://github.com/abdoelsayed2016/HKR_Dataset

ImgEdit is a large-scale, high-quality image-editing dataset comprising 1.2 million carefully curated edit pairs, which contain both novel and complex single-turn edits, as well as challenging multi-turn tasks.

LayoutBench-COCO is a diagnostic benchmark that examines layout-guided image generation models on arbitrary, unseen layouts. Unlike LayoutBench, LayoutBench-COCO consists of OOD layouts of real objects and suports zero-shot evaluation. LayoutBench-COCO measures 4 skills (Number, Position, Size, Combination), whose objects are from MS COCO. The new 'combination’ split consists of layouts with two objects in different spatial relations, and the remaining three splits are similar to those of LayoutBench. Download dataset at: https://huggingface.co/datasets/j-min/layoutbench-coco

LayoutBench-COCO is a diagnostic benchmark that examines layout-guided image generation models on arbitrary, unseen layouts. Unlike LayoutBench, LayoutBench-COCO consists of OOD layouts of real objects and suports zero-shot evaluation. LayoutBench-COCO measures 4 skills (Number, Position, Size, Combination), whose objects are from MS COCO. The new 'combination’ split consists of layouts with two objects in different spatial relations, and the remaining three splits are similar to those of LayoutBench. Download dataset at: https://huggingface.co/datasets/j-min/layoutbench-coco

LayoutBench-COCO is a diagnostic benchmark that examines layout-guided image generation models on arbitrary, unseen layouts. Unlike LayoutBench, LayoutBench-COCO consists of OOD layouts of real objects and suports zero-shot evaluation. LayoutBench-COCO measures 4 skills (Number, Position, Size, Combination), whose objects are from MS COCO. The new 'combination’ split consists of layouts with two objects in different spatial relations, and the remaining three splits are similar to those of LayoutBench. Download dataset at: https://huggingface.co/datasets/j-min/layoutbench-coco

LayoutBench-COCO is a diagnostic benchmark that examines layout-guided image generation models on arbitrary, unseen layouts. Unlike LayoutBench, LayoutBench-COCO consists of OOD layouts of real objects and suports zero-shot evaluation. LayoutBench-COCO measures 4 skills (Number, Position, Size, Combination), whose objects are from MS COCO. The new 'combination’ split consists of layouts with two objects in different spatial relations, and the remaining three splits are similar to those of LayoutBench. Download dataset at: https://huggingface.co/datasets/j-min/layoutbench-coco

This dataset is an OSN-transmitted (Online Social Network) version of the NIST dataset (https://www.nist.gov/itl/iad/mig/nimble-challenge-2017-evaluation). Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

We created a new dataset, named DFDM, with 6,450 Deepfake videos generated by different Autoencoder models. Specifically, five Autoencoder models with variations in encoder, decoder, intermediate layer, and input resolution, respectively, have been selected to generate Deepfakes based on the same input. We have observed the visible but subtle visual differences among different Deepfakes, demonstrating the evidence of model attribution artifacts.

iFakeFaceDB is a face image dataset for the study of synthetic face manipulation detection, comprising about 87,000 synthetic face images generated by the Style-GAN model and transformed with the GANprintR approach. All images were aligned and resized to the size of 224 x 224. Source: https://github.com/socialabubi/iFakeFaceDB Image Source: https://github.com/socialabubi/iFakeFaceDB

The Kvasir-VQA dataset is an extended dataset derived from the HyperKvasir and Kvasir-Instrument datasets, augmented with question-and-answer annotations. This dataset is designed to facilitate advanced machine learning tasks in gastrointestinal (GI) diagnostics, including image captioning, Visual Question Answering (VQA) and text-based generation of synthetic medical images.

LAION-COCO is the world’s largest dataset of 600M generated high-quality captions for publicly available web-images. The images are extracted from the english subset of Laion-5B with an ensemble of BLIP L/14 and 2 CLIP versions (L/14 and RN50x64). This dataset allow models to produce high quality captions for images. Source: LAION COCO: 600M SYNTHETIC CAPTIONS FROM LAION2B-EN Image Source: https://laion.ai/blog/laion-coco/

LayoutBench is a diagnostic benchmark that examines 4 spatial control skills (number, position, size, shape), where each skill consists of 2 OOD layout splits, i.e., in total 8 tasks = 4 skills x 2 splits. To disentangle spatial control from other aspects of image generation, such as generating diverse objects, LayoutBench keeps the object configurations of CLEVR, and changes the spatial layouts.

#This dataset provides the VCIP 2020 Grand Challenge on the NIR Image Colorization dataset. ##You can refer to https://jchenhkg.github.io/projects/NIR2RGB_VCIP_Challenge/ for a detailed description of this dataset. ## If you think this dataset is helpful, please feel free to cite our paper: @inproceedings{yang2023cooperative, title={Cooperative Colorization: Exploring Latent Cross-Domain Priors for NIR Image Spectrum Translation}, author={Yang, Xingxing and Chen, Jie and Yang, Zaifeng}, booktitle={Proceedings of the 31st ACM International Conference on Multimedia}, pages={2409--2417}, year={2023} }

QST contains 1,167 video clips that are cut out from 216 time-lapse 4K videos collected from YouTube, which can be used for a variety of tasks, such as (high-resolution) video generation, (high-resolution) video prediction, (high-resolution) image generation, texture generation, image inpainting, image/video super-resolution, image/video colorization, image/video animating, etc. Each short clip contains multiple frames (from a minimum of 58 frames to a maximum of 1,200 frames, a total of 285,446 frames), and the resolution of each frame is more than 1,024 x 1,024. Specifically, QST consists of a training set (containing 1000 clips, totally 244,930 frames), a validation set (containing 100 clips, totally 23,200 frames), and a testing set (containing 67 clips, totally 17,316 frames). Click here (Key: qst1) to download the QST dataset.

A screenshot-caption dataset containing 135k pairs of screenshots and captions extracted from Google Play.

SketchHairSalon is a dataset for hair generation containing thousands of annotated hair sketch-image pairs and corresponding hair mattes.

Depth vision has been recently used in many locomotion devices with the objective to ease the life of disabled people toward reaching more ecological lifestyle. This is due to the fact that such cameras are cheap, compact and can provide rich information about the environment. Our dataset provides many recordings of point cloud and other types of data during different locomotion modes in urban context. If you used this data, please cite the following papers below: 1-Depth Vision based Terrain Detection Algorithm during Human Locomotion 2-Using Depth Vision for Terrain Detection during Active Locomotion

We present a novel Animation CelebHeads dataset (AnimeCeleb) to address an animation head reenactment. Different from previous animation head datasets, we utilize 3D animation models as the controllable image samplers, which can provide a large amount of head images with their corresponding detailed pose annotations. To facilitate a data creation process, we build a semi-automatic pipeline leveraging an open 3D computer graphics software with a developed annotation system. After training with the AnimeCeleb, recent head reenactment models produce high-quality animation head reenactment results, which are not achievable with existing datasets. Furthermore, motivated by metaverse application, we propose a novel pose mapping method and architecture to tackle a cross-domain head reenactment task. During inference, a user can easily transfer one's motion to an arbitrary animation head. Experiments demonstrate the usefulness of the AnimeCeleb to train animation head reenactment models, and the superiority of our cross-domain head reenactment model compared to state-of-the-art methods. Our dataset and code are available at https://github.com/kangyeolk/AnimeCeleb.