Depth Estimation

The MUAD dataset (Multiple Uncertainties for Autonomous Driving), consisting of 10,413 realistic synthetic images with diverse adverse weather conditions (night, fog, rain, snow), out-of-distribution objects, and annotations for semantic segmentation, depth estimation, object, and instance detection. Predictive uncertainty estimation is essential for the safe deployment of Deep Neural Networks in real-world autonomous systems and MUAD allows to a better assess the impact of different sources of uncertainty on model performance.

Relative Human (RH) contains multi-person in-the-wild RGB images with rich human annotations, including: Depth layers: relative depth relationship/ordering between all people in the image. Age group classfication: adults, teenagers, kids, babies. Others: Genders, Bounding box, 2D pose.

CelebA-Spoof is a large-scale face anti-spoofing dataset recently introduced in [53]. The dataset contains 625,537 images of 10,177 celebrities captured under different spoof mediums, environments and illumination conditions. The original dataset proposes three different evaluation protocols. For our experimentation, we focus on the most general ”intra” protocol, in which different spoof types, environments and illumination conditions are used for both training and testing. To generate CelebA-Spoof-Enroll, the personalized version of the CelebA-Spoof anti-spoofing dataset, we start by setting the desired enrollment set size N. We decide for a constant number of enrollment images per user to be consistent with the implementation in typical commercial applications and to simplify the dataset definition. For both the training and test split, we count the number of live samples per user and discard those users having ≤ N live samples. So, if we desire a higher value of N, a larger number of users would get rejected and hence the number of training samples would be less. Note that it is not possible to include all original data, as a number of users in CelebA-Spoof are missing live samples. Nevertheless, only a very small percentage of training and test data is discarded through this process when choosing N < 10. For each accepted user, the first N live samples (ordered according to the ascending alphanumeric ordering of the original filenames) are chosen to define its enrollment set. The rest of the live samples and the spoof samples are marked as query samples. It is important to note that using this deterministic method of obtaining enrollment samples does not introduce unwanted bias as most of the CelebASpoof images are randomly crawled from the internet and are not ordered according to specific criteria. With this setting, we associate a user identifier with each query sample such that queries can be easily mapped to the correct enrollment set. This method of filtering users for a desired enrollment size N is performed for both training and test split. In the rest of the paper, we refer to CelebA-Spoof-EnrollN or in short CASp-EnrollN to describe the personalized version of the dataset with N enrollment images (See paper for additional details)

We present a new large-scale photorealistic panoramic dataset named FutureHouse, which has the following characteristics. 1) It contains over 70,000 high-quality models with high-resolution meshes and physical material. All models are measured in real world standards. 2) Selected scene layouts are carefully designed by over 100 excellent artists. All of selected layouts are used in realworld display. 3) It contains 28,579 good panoramic views from 1,752 house-scale scenes. Therefore, it can be used for perspective image tasks as well as omnidirectional image tasks. 4) More physical material representation. Most materials are represent by microfacet BRDF modeling metalness, and the rest are represent by special shading models, e.g., cloth material and transmission material. 5) High rendering quality. Benefiting from commercial rendering engine, Unreal engine 4, and powerful deep learning super sampling (DLSS), our renderings have less noise. Our SVBRDF representation including base color and metalness is capable of producing nonmonochrome specular reflectance.

Aa new cross-season scaleless monocular depth prediction dataset from CMU Visual Localization dataset through structure from motion. Source: SeasonDepth: Cross-Season Monocular Depth Prediction Dataset and Benchmark under Multiple Environments

SiW (Spoofing in the Wild) is a face anti-spoofing dataset recently introduced in [29] where images are extracted from short videos captured at high resolution and 30 frames per second. In total, 4,478 videos are collected from 165 subjects including variations in spoof type, recording device, illumination condition, pose and facial expression. To define train and test sets, we start by following the splitting system described as Protocol 1 in [29]. We further subsample train and test sets by extracting 1 every 10 frames (or every 0.33 seconds) in each video, since consecutive frames are almost identical. Finally, since we observe that simple models can reach very high accuracy, we further make the task harder by creating 2 separate training and evaluation folds in a way that the model is trained and tested on different spoof types. Since the original dataset is collected using two different capturing devices, we can use this information for the definition of enrollment sets. Indeed, in a real scenario, the subject would have to enroll twice when using two different devices, meaning that a unique enrollment set will be generated for each combination of subject and capturing devices. Capturing the sensor bias in the enrollment can be particularly beneficial, for example to detect if the resolution of a face changes, which can indicate a replay attack. To extract enrollment data we choose a single video from each subject and sensor among the available ones. We arbitrarily pick the video without illumination changes and with variation in subject pose, as it resembles the enrollment conditions required in real devices. We then equidistantly sample N frames over the video to construct the enrollment set. As shown in Fig. 2, this allows capturing different poses and facial expressions from the subject. Finally, we exclude all the frames in this video from training and test data to avoid in341 formation leakage and match all the remaining queries from the same subject and sensor to their unique enrollment set. (See paper for additional details)

The UASOL an RGB-D stereo dataset, that contains 160902 frames, filmed at 33 different scenes, each with between 2 k and 10 k frames. The frames show different paths from the perspective of a pedestrian, including sidewalks, trails, roads, etc. The images were extracted from video files with 15 fps at HD2K resolution with a size of 2280 × 1282 pixels. The dataset also provides a GPS geolocalization tag for each second of the sequences and reflects different climatological conditions. It also involved up to 4 different persons filming the dataset at different moments of the day. We propose a train, validation and test split to train the network. Additionally, we introduce a subset of 676 pairs of RGB Stereo images and their respective depth, which we extracted randomly from the entire dataset. This given test set is introduced to make comparability possible between the different methods trained with the dataset.

A synthetic depth estimation dataset for benchmark rendered from a high-quality CAD indoor environment - About 3.5K RGBD pairs with left-right stereo - Challenging viewing direction - Challenging different light condition

This dataset presents a vision and perception research dataset collected in Rome, featuring RGB data, 3D point clouds, IMU, and GPS data. We introduce a new benchmark targeting visual odometry and SLAM, to advance the research in autonomous robotics and computer vision. This work complements existing datasets by simultaneously addressing several issues, such as environment diversity, motion patterns, and sensor frequency. It uses up-to-date devices and presents effective procedures to accurately calibrate the intrinsic and extrinsic of the sensors while addressing temporal synchronization. During recording, we cover multi-floor buildings, gardens, urban and highway scenarios. Combining handheld and car-based data collections, our setup can simulate any robot (quadrupeds, quadrotors, autonomous vehicles). The dataset includes an accurate 6-dof ground truth based on a novel methodology that refines the RTK-GPS estimate with LiDAR point clouds through Bundle Adjustment. All sequences divided in training and testing are accessible through our website.

4D Light Field Dataset is a light field benchmark consisting of 24 carefully designed synthetic, densely sampled 4D light fields with highly accurate disparity ground truth.

A real-world dataset, with hyper-accurate digital counterpart & comprehensive ground-truth annotation. Dataset Content - 200 sequences (~400 mins) - 398 objects (324 stationary, 74 dynamic) - 2 real indoor scenes - Single + multi-user activities Sensor Data per device - 2 x outward-facing monochrome camera streams - 1 x outward-facing RGB camera stream - 2 x IMU streams - 2 x Internal-facing eye tracking cameras - Complete sensor calibrations

DRACO20K dataset is used for evaluating object canonicalization on methods that estimate a canonical frame from a monocular input image. Provides: 1. Mixed Reality Multi-view RGB-D images rendered from ShapeNet objects 2. Camera poses 3. NOCS maps 4. Semantic 2D keypoints with visibility 5. Object-centric mask

A new synthetic, multi-purpose dataset - called ENRICH - for testing photogrammetric and computer vision algorithms. Compared to existing datasets, ENRICH offers higher resolution images also rendered with different lighting conditions, camera orientation, scales, and field of view. Specifically, ENRICH is composed of three sub-datasets: ENRICH-Aerial, ENRICH-Square, and ENRICH-Statue, each exhibiting different characteristics. The proposed dataset is useful for several photogrammetry and computer vision-related tasks, such as the evaluation of hand-crafted and deep learning-based local features, effects of ground control points (GCPs) configuration on the 3D accuracy, and monocular depth estimation. Each zip file in the root is relative to a specific dataset: - ENRICH-Aerial, is an aerial image block of the city of Launceston, Australia. The acquisition is performed by simulating a typical oblique aerial camera with five views (nadir and four oblique views). - ENRICH-Square, is a ground-level dataset of a square captured by four cameras, each one moving on a different path with different focal length, orientation, and lighting conditions. - ENRICH-Statue, is a ground-level dataset portraying a statue (placed in the center of the ENRICH-Square scene), acquired using four cameras moving on different paths with different focal lengths, orientations, and lighting conditions. Be sure to check the README file in the dataset root for information on folder structure and file contents. Please refer to the related paper (https://doi.org/10.1016/j.isprsjprs.2023.03.002) for information about the generation method and the purpose of ENRICH.

General-purpose Visual Understanding Evaluation (G-VUE) is a comprehensive benchmark covering the full spectrum of visual cognitive abilities with four functional domains -- Perceive, Ground, Reason, and Act. The four domains are embodied in 11 carefully curated tasks, from 3D reconstruction to visual reasoning and manipulation. Source: Perceive, Ground, Reason, and Act: A Benchmark for General-purpose Visual Representation Image Source: https://github.com/wllmzhu/G-VUE

HAMMER dataset contains 13 Scenes. Each scene has two setups, with/without objects (with : scene includes several objects with various surface material, without : scene with only backgrounds - naked) and each scene has two camera trajectories. Each trajectories composed with roughly 300 frames, which adds up to 16k frames in total (13 x 2 x 2 x 300). Each trajectory contains corresponding images from each cameras : d435 – stereo, l515 – Lidar (D-ToF), polarization – RGBP (RGB with polarization), tof – (I-ToF). Each camera folder contains its intrinsic file and its own recorded images together with rendered depth GT / instance GT and camera pose. All the cameras are fully synchronized via robotic arm’s data acquisition setup. Source: On the Importance of Accurate Geometry Data for Dense 3D Vision Tasks Image Source: On the Importance of Accurate Geometry Data for Dense 3D Vision Tasks

This dataset contains panoramic video captured from a helmet-mounted camera while riding a bike through suburban Northern Virginia. Source: Headcam

A Simulated Benchmark for multi-modal SLAM Systems Evaluation in Large-scale Dynamic Environments.

High Quality Indoor Monocular Depth Estimation Dataset with focus on performance variation across space type - 1260 high quality evaluation pairs - Detailed inference variance across space types - Additional indoor image and depth pairs for training

InstaOrder can be used to understand the geometrical relationships of instances in an image. The dataset consists of 2.9M annotations of geometric orderings for class-labeled instances in 101K natural scenes. The scenes were annotated by 3,659 crowd-workers regarding (1) occlusion order that identifies occluder/occludee and (2) depth order that describes ordinal relations that consider relative distance from the camera.

MagicBathyNet is a benchmark dataset made up of image patches of Sentinel-2, SPOT-6 and aerial imagery, bathymetry in raster format and seabed classes annotations. Dataset also facilitates unsupervised learning for model pre-training in shallow coastal areas. MagicBathyNet contains 3355 RGB co-registered triplets of Sentinel-2 (S2), SPOT-6, and aerial image patches, complemented by 1244 RGB co-registered S2 and SPOT-6 doublets, 3354 DSM (Digital Surface Model) raster patches for the aerial patches and 3396 DSM raster patches for S2 and SPOT-6. Additionally, it contains 533 annotated raster patches for seabed habitat and type, facilitating supervised pixel-based classification. Each patch covers 180x180m, represented by 18x18 pixels in S2 imagery, 30x30 pixels in SPOT-6 imagery and 720x720 pixels in airborne imagery.

Mila Simulated Floods Dataset is a 1.5 square km virtual world using the Unity3D game engine including urban, suburban and rural areas. The urban environment contains skyscrapers, large buildings, and roads, as well as objects such as traffic items and vehicles. The rural environment consists of a landscape of grassy hills, forests, and mountains, with sparse houses and other buildings such as a church, and no roads. The rural and urban areas make up for 1 square km of our virtual world. The suburban environment is a residential area of 0.5 square km with many individual houses with front yards. To gather the simulated dataset, we captured before and after flood pairs from 2000 viewpoints with the following modalities: - non-flooded RGB image, depth map, segmentation map - flooded RGB image, binary mask of the flooded area, segmentation map The camera was placed about 1.5 above ground, and has a field of view of 120 degree, and the resolution of the images is 1200 x 900. At each viewpoint, we took 10 pictures, by varying slightly the position of the camera in order to augment the dataset.

Pano3D is a new benchmark for depth estimation from spherical panoramas. Its goal is to drive progress for this task in a consistent and holistic manner. The Pano3D 360 depth estimation benchmark provides a standard Matterport3D train and test split, as well as a secondary GibsonV2 partioning for testing and training as well. The latter is used for zero-shot cross dataset transfer performance assessment and decomposes it into 3 different splits, each one focusing on a specific generalization axis.

SuHiFiMask (Surveillance High-Fidelity Mask) extends FAS to real surveillance scenes rather than mimicking low-resolution images and surveillance environments. It contains 10,195 videos from 101 subjects of different age groups, which are collected by 7 mainstream surveillance cameras. Source: Surveillance Face Presentation Attack Detection Challenge

SuperCaustics is a simulation tool made in Unreal Engine for generating massive computer vision datasets that include transparent objects. SuperCaustics is a real-time, open-source simulation of transparent objects designed for deep learning applications. SuperCaustics features extensive modules for stochastic environment creation; uses hardware ray-tracing to support caustics, dispersion, and refraction; and enables generating massive datasets with multi-modal, pixel-perfect ground truth annotations. <p align="center"> <img src="https://github.com/MMehdiMousavi/SuperCaustics/raw/main/Assets/SuperCaustics.gif" alt="drawing" width="600"/> </p>

Comprised of real human and wax figure images and videos that endorse the problem of face spoofing detection. The dataset consists of more than 1800 face images and 110 videos of 55 people/waxworks, arranged in training, validation and test sets with a large range in expression, illumination and pose variations. Source: The SWAX Benchmark: Attacking Biometric Systems with Wax Figures

The Autonomous-driving StreAming Perception (ASAP) benchmark is a benchmark to evaluate the online performance of vision-centric perception in autonomous driving. It extends the 2Hz annotated nuScenes dataset by generating high-frame-rate labels for the 12Hz raw images. Source: Are We Ready for Vision-Centric Driving Streaming Perception? The ASAP Benchmark Image Source: https://github.com/jeffwang987/asap

57 stock videos from Pexels, predominantly covering road scenes which involve minimal distortion. They involve different camera setups, also with varying camera heights, obstacles present throughout some videos (for e.g. car hood), highly varying image resolutions, and even weather conditions (day, rain, snow, night etc.). For more details, check the paper CamLessMonoDepth: Monocular Depth Estimation with Unknown Camera Parameters https://arxiv.org/pdf/2110.14347v1.pdf

This dataset contains synthetic images extracted from the CARLA simulator along with rich information extracted from the deferred rendering pipeline of Unreal Engine 4. The main purpose of this dataset is the training of the state-of-the-art image-to-image translation model proposed by Intel Labs "Enhancing Photorealism Enhancement" (EPE). Translation results derived from the model targeting the characteristics of Cityscapes, KITTI, and Mapillary Vistas are also provided. Computer vision-based models trained on these data are expected to perform better when deployed in the real world.

This dataset provides simulated flood inundation maps of Abu Dhabi's coast under 174 different shoreline protection scenarios. The maps were produced with a high-fidelity physics-based hydrodynamic simulator under a 0.5-meter sea level rise projection. The details of the hydrodynamic model are reported in [1]. The coastline was partitioned into 17 segments. The input protection scenarios were encoded as 17-dimensional binary vectors, with 1 indicating the placement of containments and 0 otherwise. The output inundation depth maps are named according to protection scenarios. Each map is a 1024x1024 matrix containing the estimated peak water level values (in meters) for selected 12066 locations along the coastline of Abu Dhabi. The mapping from gird indices to the original geographical coordinates of these locations is provided in a separate file (ad_grid_1024.npy). A Jupyter notebook with a sample Python code is included for visualizing the data entries (see Data_Visualization.ipynb). [1] Chow, Aaron C. H., and Jiayun Sun. 2022. "Combining Sea Level Rise Inundation Impacts, Tidal Flooding and Extreme Wind Events along the Abu Dhabi Coastline" Hydrology 9, no. 8: 143. https://doi.org/10.3390/hydrology9080143

ConSLAM is a real-world dataset collected periodically on a construction site to measure the accuracy of mobile scanners' SLAM algorithms. The dataset contains time-synchronized and spatially registered RGB and NIR images and 360-deg LiDAR scans, 9-axis IMU measurements, and professional ground-truth terrestrial laser scans. This dataset reflects the periodic need to scan construction sites with the aim of accurately monitoring progress using a hand-held scanner. The sensors used for data acquisition are: - LiDAR: Velodyne VLP-16. - RGB camera: Alvium U-319c, 3.2 MP. - NIR camera: Alvium 1800 U-501, 5.0 MP. - 9-axis IMU: Xsens MTi-610.

Provide: * 10 pickle files * 8 pickle files are used to generate depth maps * 2 pickle files are data of fronto parallel texture with their ground truth depth * Each pickle file contains the parameter of the camera system (aperture size, optical power, flange focal distance...)

A dataset of 100K synthetic images of skin lesions, ground-truth (GT) segmentations of lesions and healthy skin, GT segmentations of seven body parts (head, torso, hips, legs, feet, arms and hands), and GT binary masks of non-skin regions in the texture maps of 215 scans from the 3DBodyTex.v1 dataset [2], [3] created using the framework described in [1]. The dataset is primarily intended to enable the development of skin lesion analysis methods. Synthetic image creation consisted of two main steps. First, skin lesions from the Fitzpatrick 17k dataset were blended onto skin regions of high-resolution three-dimensional human scans from the 3DBodyTex dataset [2], [3]. Second, two-dimensional renders of the modified scans were generated. Use of the dataset, in part or in full, is conditional on citation of the following work: [1] Ashish Sinha, Jeremy Kawahara, Arezou Pakzad, Kumar Abhishek, Matthieu Ruthven, Enjie Ghorbel, Anis Kacem, Djamila Aouada, Ghassan Hamarneh, ‘DermSynth3D: Synthesis of in-the-wild annotated dermatology images’, 2023. [2] A. Saint, A. E. Rahman Shabayek, K. Cherenkova, G. Gusev, D. Aouada, and B. Ottersten, ‘Bodyfitr: Robust Automatic 3D Human Body Fitting’, in 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan: IEEE, Sep. 2019, pp. 484–488. doi: 10.1109/ICIP.2019.8803819. [3] A. Saint et al., ‘3DBodyTex: Textured 3D Body Dataset’, in 2018 International Conference on 3D Vision (3DV), Verona: IEEE, Sep. 2018, pp. 495–504. doi: 10.1109/3DV.2018.00063.

EUEN17037 Daylight and View Standard Test Dataset.

We provide all the expected data inputs to GUISS such as meshes, texture images, and blend files. Generated datasets used in our experiments along with the stereo depth estimations can be downloaded. We have defined seven dataset types: scene_reconstructions, texture_variation, gaea_texture_variation, generative_texture, terrain_variation, rocks, and generative_texture_snow. Each dataset type contains renderings with varying values of different parameters such as lighting angle, texture imgs, albedo, etc. Position each dataset type folder under data/dataset/. Details and links at: https://github.com/nasa-jpl/guiss

The dfd_indoor dataset contains 110 images for training and 29 images for testing. The dfd_outdoor dataset contains 34 images for tests; no ground truth was given for this dataset, as the depth sensor only works on indoor scenes. Source: DFD dataset

InfraParis is a novel and versatile dataset supporting multiple tasks across three modalities: RGB, depth, and infrared. From the city to the suburbs, it contains a variety of styles in different areas of the greater Paris area, providing rich semantic information. InfraParis contains 7301 images with bounding boxes and full semantic (19 classes) annotations. We assess various state-of-the-art baseline techniques, encompassing models for the tasks of semantic segmentation, object detection, and depth estimation.