3D Face Reconstruction

Year after year, the demand for ever-better smartphone photos continues to grow, in particular in the domain of portrait photography. Manufacturers thus use perceptual quality criteria throughout the development of smartphone cameras. This costly procedure can be partially replaced by automated learning-based methods for image quality assessment (IQA). Due to its subjective nature, it is necessary to estimate and guarantee the consistency of the IQA process, a characteristic lacking in the mean opinion scores (MOS) widely used for crowdsourcing IQA. In addition, existing blind IQA (BIQA) datasets pay little attention to the difficulty of cross-content assessment, which may degrade the quality of annotations. This paper introduces PIQ23, a portrait-specific IQA dataset of 5116 images of 50 predefined scenarios acquired by 100 smartphones, covering a high variety of brands, models, and use cases. The dataset includes individuals of various genders and ethnicities who have given explicit and informed consent for their photographs to be used in public research. It is annotated by pairwise comparisons (PWC) collected from over 30 image quality experts for three image attributes: face detail preservation, face target exposure, and overall image quality. An in-depth statistical analysis of these annotations allows us to evaluate their consistency over PIQ23. Finally, we show through an extensive comparison with existing baselines that semantic information (image context) can be used to improve IQA predictions. The dataset along with the proposed statistical analysis and BIQA algorithms are available: https://github.com/DXOMARKResearch/PIQ2023

ROF is a dataset for occluded face recognition that contains faces with both upper face occlusion, due to sunglasses, and lower face occlusion, due to masks.

Description: 1,995 People Face Images Data (Asian race). For each subject, more than 20 images per person with frontal face were collected. This data can be used for face recognition and other tasks. Data size: 1,995 people, more than 20 images per person with frontal face Race distribution: Asian people

The COVID-19 pandemic raises the problem of adapting face recognition systems to the new reality, where people may wear surgical masks to cover their noses and mouths. Traditional data sets (e.g., CelebA, CASIA-WebFace) used for training these systems were released before the pandemic, so they now seem unsuited due to the lack of examples of people wearing masks. We propose a method for enhancing data sets containing faces without masks by creating synthetic masks and overlaying them on faces in the original images. Our method relies on Spark AR Studio, a developer program made by Facebook that is used to create Instagram face filters. In our approach, we use 9 masks of different colors, shapes and fabrics. We employ our method to generate a number of 445,446 (90%) samples of masks for the CASIA-WebFace data set.

The Cat Facial Landmarks in the Wild (CatFLW) dataset contains 2079 images of cats' faces in various environments and conditions, annotated with 48 facial landmarks and a bounding box on the cat’s face.

The COVID-19 pandemic raises the problem of adapting face recognition systems to the new reality, where people may wear surgical masks to cover their noses and mouths. Traditional data sets (e.g., CelebA, CASIA-WebFace) used for training these systems were released before the pandemic, so they now seem unsuited due to the lack of examples of people wearing masks. We propose a method for enhancing data sets containing faces without masks by creating synthetic masks and overlaying them on faces in the original images. Our method relies on Spark AR Studio, a developer program made by Facebook that is used to create Instagram face filters. In our approach, we use 9 masks of different colors, shapes and fabrics. We employ our method to generate a number of 196,254 (96.8%) masks for the CelebA data set.

CI-MNIST (Correlated and Imbalanced MNIST) is a variant of MNIST dataset with introduced different types of correlations between attributes, dataset features, and an artificial eligibility criterion. For an input image $x$, the label $y \in \\{1, 0\\}$ indicates eligibility or ineligibility, respectively, given that $x$ is even or odd. The dataset defines the background colors as the protected or sensitive attribute $s \in \\{0, 1\\}$, where blue denotes the unprivileged group and red denotes the privileged group. The dataset was designed in order to evaluate bias-mitigation approaches in challenging setups and be capable of controlling different dataset configurations.

The DCM dataset is composed of 772 annotated images from 27 golden age comic books. We freely collected them from the free public domain collection of digitized comic books Digital Comics Museum. One album per available publisher was selected to get as many different styles as possible. We made ground-truth bounding boxes of all panels, all characters (body + faces), small or big, human-like or animal-like. Image source: https://gitlab.univ-lr.fr/crigau02/dcm-dataset/-/tree/master

The MERL-RAV (MERL Reannotation of AFLW with Visibility) Dataset contains over 19,000 face images in a full range of head poses. Each face is manually labeled with the ground-truth locations of 68 landmarks, with the additional information of whether each landmark is unoccluded, self-occluded (due to extreme head poses), or externally occluded. The images were annotated by professional labelers, supervised by researchers at Mitsubishi Electric Research Laboratories (MERL).

The MLFP dataset consists of face presentation attacks captured with seven 3D latex masks and three 2D print attacks. The dataset contains videos captured from color, thermal and infrared channels. Source: Learning One Class Representations for Face Presentation Attack Detection using Multi-channel Convolutional Neural Networks Image Source: http://iab-rubric.org/papers/2017_cvprw_18.pdf

BAFMD contains images posted on Twitter during the pandemic from around the world with more images from underrepresented race and age groups to mitigate the problem for the face mask detection task. Source: Bias-Aware Face Mask Detection Dataset Image Source: https://arxiv.org/pdf/2211.01207v1.pdf

CASIA-Face-Africa is a face image database which contains 38,546 images of 1,183 African subjects. Multi-spectral cameras are utilized to capture the face images under various illumination settings. Demographic attributes and facial expressions of the subjects are also carefully recorded. For landmark detection, each face image in the database is manually labeled with 68 facial keypoints. A group of evaluation protocols are constructed according to different applications, tasks, partitions and scenarios. The proposed database along with its face landmark annotations, evaluation protocols and preliminary results form a good benchmark to study the essential aspects of face biometrics for African subjects, especially face image preprocessing, face feature analysis and matching, facial expression recognition, sex/age estimation, ethnic classification, face image generation, etc.

CelebA-Spoof is a large-scale face anti-spoofing dataset recently introduced in [53]. The dataset contains 625,537 images of 10,177 celebrities captured under different spoof mediums, environments and illumination conditions. The original dataset proposes three different evaluation protocols. For our experimentation, we focus on the most general ”intra” protocol, in which different spoof types, environments and illumination conditions are used for both training and testing. To generate CelebA-Spoof-Enroll, the personalized version of the CelebA-Spoof anti-spoofing dataset, we start by setting the desired enrollment set size N. We decide for a constant number of enrollment images per user to be consistent with the implementation in typical commercial applications and to simplify the dataset definition. For both the training and test split, we count the number of live samples per user and discard those users having ≤ N live samples. So, if we desire a higher value of N, a larger number of users would get rejected and hence the number of training samples would be less. Note that it is not possible to include all original data, as a number of users in CelebA-Spoof are missing live samples. Nevertheless, only a very small percentage of training and test data is discarded through this process when choosing N < 10. For each accepted user, the first N live samples (ordered according to the ascending alphanumeric ordering of the original filenames) are chosen to define its enrollment set. The rest of the live samples and the spoof samples are marked as query samples. It is important to note that using this deterministic method of obtaining enrollment samples does not introduce unwanted bias as most of the CelebASpoof images are randomly crawled from the internet and are not ordered according to specific criteria. With this setting, we associate a user identifier with each query sample such that queries can be easily mapped to the correct enrollment set. This method of filtering users for a desired enrollment size N is performed for both training and test split. In the rest of the paper, we refer to CelebA-Spoof-EnrollN or in short CASp-EnrollN to describe the personalized version of the dataset with N enrollment images (See paper for additional details)

CUHK Face Sketch database (CUFS) is for research on face sketch synthesis and face sketch recognition. It includes 188 faces from the Chinese University of Hong Kong (CUHK) student database, 123 faces from the AR database [1], and 295 faces from the XM2VTS database [2]. There are 606 faces in total. For each face, there is a sketch drawn by an artist based on a photo taken in a frontal pose, under normal lighting condition, and with a neutral expression. [1] A. M. Martinez, and R. Benavente, “The AR Face Database,” CVC Technical Report #24, June 1998. [2] K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre, “XM2VTSDB: the Extended of M2VTS Database,” in Proceedings of International Conference on Audio- and Video-Based Person Authentication, pp. 72-77, 1999. Source Paper: X. Wang and X. Tang, “Face Photo-Sketch Synthesis and Recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 31, 2009 Image Source: CUHK Face Sketch Database (CUFS) Source: CUHK Face Sketch Database (CUFS)

DAD-3DHeads dataset consists of 44,898 images collected from various sources (37,840 in the training set, 4,312 in the validation set, and 2,746 in the test set). The dataset is well-balanced over a wide range of poses, face expressions, and occlusions. The attribute labels are a valuable signal for subgroup analysis and for generalisation to in-the-wild deployment conditions.

We created a new dataset, named DFDM, with 6,450 Deepfake videos generated by different Autoencoder models. Specifically, five Autoencoder models with variations in encoder, decoder, intermediate layer, and input resolution, respectively, have been selected to generate Deepfakes based on the same input. We have observed the visible but subtle visual differences among different Deepfakes, demonstrating the evidence of model attribution artifacts.

FEAFA+ is a dataset for Facial expression analysis and 3D Facial animation. It includes 150 video sequences from FEAFA and DISFA, with a total of 230,184 frames being manually annotated on floating-point intensity value of 24 redefined AUs using the Expression Quantitative Tool.

Florence 4D is a dataset that consists of dynamic sequences of 3D face models, where a combination of synthetic and real identities exhibit an unprecedented variety of 4D facial expressions, with variations that include the classical neutral-apex transition, but generalize to expression-to-expression. It is designed for research in 4D facial analysis, with a particular focus on dynamic expressions.

iFakeFaceDB is a face image dataset for the study of synthetic face manipulation detection, comprising about 87,000 synthetic face images generated by the Style-GAN model and transformed with the GANprintR approach. All images were aligned and resized to the size of 224 x 224. Source: https://github.com/socialabubi/iFakeFaceDB Image Source: https://github.com/socialabubi/iFakeFaceDB

Paper Abstract We present IJB–S dataset, an open-source IARPA Janus Surveillance Video Benchmark and associated protocols. The dataset consists of images and surveillance video collected from 202 subjects at a Department of Defense (DoD) training facility. Surveillance video was captured across multiple vignettes representative of a variety of real-world surveillance use cases that are particularly of interest to law enforcement and national security communities. Each video was annotated by human subject matter experts in order to generate ground truth identity and bounding box face labels. In total, over 10 million annotations were collected for the dataset.

We have cleaned the noisy IMDB-WIKI dataset using a constrained clustering method, resulting this new benchmark for in-the-wild age estimation. The annotations also allow this dataset to use for some other tasks, like gender classification and face recognition/verification. For more details, please refer to our FPAge paper.

The LAGENDA dataset is a large-scale dataset with age and gender annotations for face and body bounding boxes. The dataset consists of 67,159 images from the Open Images Dataset and comprises 84,192 pairs (FaceCrop, BodyCrop). This dataset offers a high level of diversity, encompassing various scenes and domains. It contains minimal celebrity data, thus reflecting real-world, in-the-wild scenarios. The dataset spans a wide age range, from 0 to 95 years old.

Large Age-Gap (LAG) is a dataset for face verification, The dataset contains 3,828 images of 1,010 celebrities. For each identity at least one child/young image and one adult/old image are present.

The dataset contains the annotations of characters' visual appearances, in the form of tracks of face bounding boxes, and the associations with characters' textual mentions, when available. The detection and annotation of the visual appearances of characters in each video clip of each movie was achieved through a semi-automatic approach. The released dataset contains more than 24k annotated video clips, including 63k visual tracks and 34k textual mentions, all associated with their character identities. Source: M-VAD Names Dataset

Multimodal Dyadic Behavior (MMDB) dataset is a unique collection of multimodal (video, audio, and physiological) recordings of the social and communicative behavior of toddlers. The MMDB contains 160 sessions of 3-5 minute semi-structured play interaction between a trained adult examiner and a child between the age of 15 and 30 months. The MMDB dataset supports a novel problem domain for activity recognition, which consists of the decoding of dyadic social interactions between adults and children in a developmental context. Source: MMDB

The Surrey Audio-Visual Expressed Emotion (SAVEE) dataset was recorded as a pre-requisite for the development of an automatic emotion recognition system. The database consists of recordings from 4 male actors in 7 different emotions, 480 British English utterances in total. The sentences were chosen from the standard TIMIT corpus and phonetically-balanced for each emotion. The data were recorded in a visual media lab with high quality audio-visual equipment, processed and labeled. To check the quality of performance, the recordings were evaluated by 10 subjects under audio, visual and audio-visual conditions. Classification systems were built using standard features and classifiers for each of the audio, visual and audio-visual modalities, and speaker-independent recognition rates of 61%, 65% and 84% achieved respectively.

SiW (Spoofing in the Wild) is a face anti-spoofing dataset recently introduced in [29] where images are extracted from short videos captured at high resolution and 30 frames per second. In total, 4,478 videos are collected from 165 subjects including variations in spoof type, recording device, illumination condition, pose and facial expression. To define train and test sets, we start by following the splitting system described as Protocol 1 in [29]. We further subsample train and test sets by extracting 1 every 10 frames (or every 0.33 seconds) in each video, since consecutive frames are almost identical. Finally, since we observe that simple models can reach very high accuracy, we further make the task harder by creating 2 separate training and evaluation folds in a way that the model is trained and tested on different spoof types. Since the original dataset is collected using two different capturing devices, we can use this information for the definition of enrollment sets. Indeed, in a real scenario, the subject would have to enroll twice when using two different devices, meaning that a unique enrollment set will be generated for each combination of subject and capturing devices. Capturing the sensor bias in the enrollment can be particularly beneficial, for example to detect if the resolution of a face changes, which can indicate a replay attack. To extract enrollment data we choose a single video from each subject and sensor among the available ones. We arbitrarily pick the video without illumination changes and with variation in subject pose, as it resembles the enrollment conditions required in real devices. We then equidistantly sample N frames over the video to construct the enrollment set. As shown in Fig. 2, this allows capturing different poses and facial expressions from the subject. Finally, we exclude all the frames in this video from training and test data to avoid in341 formation leakage and match all the remaining queries from the same subject and sensor to their unique enrollment set. (See paper for additional details)

The UFPR-Periocular dataset has 16,830 images of both eyes (33,660 cropped images of each eye) from 1,122 subjects (2,244 classes). All the images were captured by the participant using their own smartphone through a mobile application (app) developed by the authors. There are 15 samples from each subject's eye, obtained in 3 sessions (5 images per session) with a minimum interval of 8 hours between the sessions. The images were collected from June 2019 to January 2020 and have several resolutions varying from 360×160 to 1862×1008 pixels – depending on the mobile device used to capture the image. In total, the dataset has images captured from 196 different mobile devices. Each subject captured their images using the same device model. This dataset's main intra- and inter-class variability are caused by lighting variation, occlusion, specular reflection, blur, motion blur, eyeglasses, off-angle, eye-gaze, makeup, and facial expression. The authors manually annotated the eye corner of all images with 4 points (inside and outside eye corners) and used it to normalize the periocular region regarding scale and rotation. All the original and cropped periocular images, eye-corner annotations, and experimental protocol files are publicly available for the research community (upon request). The paper contains information about images' distributions by gender, age, resolution, and other experiments' details and benchmarks. Source: A new periocular dataset collected by mobile devices in unconstrained scenarios

Urban is one of the most widely used hyperspectral data used in the hyperspectral unmixing study. There are 307x307 pixels, each of which corresponds to a 2x2 m2 area. In this image, there are 210 wavelengths ranging from 400 nm to 2500 nm, resulting in a spectral resolution of 10 nm. After the channels 1-4, 76, 87, 101-111, 136-153 and 198-210 are removed (due to dense water vapor and atmospheric effects), we remain 162 channels (this is a common preprocess for hyperspectral unmixing analyses). There are three versions of ground truth, which contain 4, 5 and 6 endmembers respectively, which are introduced in the ground truth. Linda S. Kalman and Edward M. Bassett III "Classification and material identification in an urban environment using HYDICE hyperspectral data", Proc. SPIE 3118, Imaging Spectrometry III, (31 October 1997); https://doi.org/10.1117/12.283843 Hosted at: - https://rslab.ut.ac.ir/data - http://lesun.weebly.com/hyperspectral-data-set.html - https://erdc-library.erdc.dren.mil/jspui/handle/11681/2925

Description： 5,011 Images – Human Frontal face Data (Male). The data diversity includes multiple scenes, multiple ages and multiple races. This dataset includes 2,004 Caucasians , 3,007 Asians. This dataset can be used for tasks such as face detection, race detection, age detection, beard category classification. Data size： 5,011 people, one image per person Race distribution： 2,004 Caucasians , 3,007 Asians

We present a novel Animation CelebHeads dataset (AnimeCeleb) to address an animation head reenactment. Different from previous animation head datasets, we utilize 3D animation models as the controllable image samplers, which can provide a large amount of head images with their corresponding detailed pose annotations. To facilitate a data creation process, we build a semi-automatic pipeline leveraging an open 3D computer graphics software with a developed annotation system. After training with the AnimeCeleb, recent head reenactment models produce high-quality animation head reenactment results, which are not achievable with existing datasets. Furthermore, motivated by metaverse application, we propose a novel pose mapping method and architecture to tackle a cross-domain head reenactment task. During inference, a user can easily transfer one's motion to an arbitrary animation head. Experiments demonstrate the usefulness of the AnimeCeleb to train animation head reenactment models, and the superiority of our cross-domain head reenactment model compared to state-of-the-art methods. Our dataset and code are available at https://github.com/kangyeolk/AnimeCeleb.

At RSNA 2017 there was a contest to correctly identify the age of a child from an X-ray of their hand.

Celeb-HQ Face Gender Recognition Dataset * This dataset is curated for the face gender classification task. * The dataset contains 30,000 images. * There are 23,999 train images. * There are 6,001 test images. * The whole face images are divided into two classes. * There are 11,057 male images. * There are 18,943 female images.

Celeb-HQ Facial Identity Recognition Dataset * This dataset is curated for the facial identity classification task. * There are 307 identities (celebrities). * Each identity has 15 or more images. * The dataset contains 5,478 images. * There are 4,263 training images. * There are 1,215 test images.

DARK FACE dataset provides 6,000 real-world low light images captured during the nighttime, at teaching buildings, streets, bridges, overpasses, parks etc., all labeled with bounding boxes for of human face, as the main training and/or validation sets. We also provide 9,000 unlabeled low-light images collected from the same setting. Additionally, we provided a unique set of 789 paired low-light/normal-light images captured in controllable real lighting conditions (but unnecessarily containing faces), which can be used as parts of the training data at the participants' discretization. There will be a hold-out testing set of 4,000 low-light images, with human face bounding boxes annotated.

EDFace-Celeb-1M is a public Ethnically Diverse Face dataset which is used to benchmark the task of face hallucination. The dataset includes 1.7 million photos that cover different countries, with balanced race composition.