Depth Estimation

The ReDWeb dataset consists of 3600 RGB-RD image pairs collected from the Web. This dataset covers a wide range of scenes and features various non-rigid objects.

Collects high quality 360 datasets with ground truth depth annotations, by re-using recently released large scale 3D datasets and re-purposing them to 360 via rendering. Source: OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas

The Replay-Mobile Database for face spoofing consists of 1190 video clips of photo and video attack attempts to 40 clients, under different lighting conditions. These videos were recorded with current devices from the market -- an iPad Mini2 (running iOS) and a LG-G4 smartphone (running Android). This Database was produced at the Idiap Research Institute (Switzerland) within the framework of collaboration with Galician Research and Development Center in Advanced Telecommunications - Gradiant (Spain). Source: Replay-Mobile Image Source: https://core.ac.uk/download/pdf/148024307.pdf

HiFiMask is a large-scale High-Fidelity Mask dataset, namely CASIA-SURF HiFiMask (briefly HiFiMask). It contains a total amount of 54,600 videos are recorded from 75 subjects with 225 realistic masks by 7 new kinds of sensors.

The KITTI-Depth dataset includes depth maps from projected LiDAR point clouds that were matched against the depth estimation from the stereo cameras. The depth images are highly sparse with only 5% of the pixels available and the rest is missing. The dataset has 86k training images, 7k validation images, and 1k test set images on the benchmark server with no access to the ground truth. Source: Confidence Propagation through CNNs for Guided Sparse Depth Regression Image Source: http://www.cvlibs.net/datasets/kitti/eval_depth.php?benchmark=depth_prediction

This dataset accompanies our paper on synthesizing the 3D Ken Burns effect from a single image. It consists of 134041 captures from 32 virtual environments where each capture consists of 4 views. Each view contains color-, depth-, and normal-maps at a resolution of 512x512 pixels.

Detecting vehicles and representing their position and orientation in the three dimensional space is a key technology for autonomous driving. Recently, methods for 3D vehicle detection solely based on monocular RGB images gained popularity. In order to facilitate this task as well as to compare and drive state-of-the-art methods, several new datasets and benchmarks have been published. Ground truth annotations of vehicles are usually obtained using lidar point clouds, which often induces errors due to imperfect calibration or synchronization between both sensors. To this end, we propose Cityscapes 3D, extending the original Cityscapes dataset with 3D bounding box annotations for all types of vehicles. In contrast to existing datasets, our 3D annotations were labeled using stereo RGB images only and capture all nine degrees of freedom. This leads to a pixel-accurate reprojection in the RGB image and a higher range of annotations compared to lidar-based approaches. In order to ease multitask learning, we provide a pairing of 2D instance segments with 3D bounding boxes. In addition, we complement the Cityscapes benchmark suite with 3D vehicle detection based on the new annotations as well as metrics presented in this work. Dataset and benchmark are available online.

The Web Stereo Video Dataset consists of 553 stereoscopic videos from YouTube. This dataset has a wide variety of scene types, and features many nonrigid objects.

An in-the-wild stereo image dataset, comprising 49,368 image pairs contributed by users of the Holopix mobile social platform. Source: Holopix50k: A Large-Scale In-the-wild Stereo Image Dataset

The HRWSI dataset consists of about 21K diverse high-resolution RGB-D image pairs derived from the Web stereo images. Also, it provides sky segmentation masks, instance segmentation masks as well as invalid pixel masks.

Depth in the Wild is a dataset for single-image depth perception in the wild, i.e., recovering depth from a single image taken in unconstrained settings. It consists of images in the wild annotated with relative depth between pairs of random points.

RGB-D-D is a large-scale dataset for depth map super-resolution (SR). It consists of real-world paired low-resolution (LR) and high-resolution (HR) depth maps. The paired LR and HR depth maps are captured from mobile phone and Lucid Helios respectively ranging from indoor scenes to challenging outdoor scenes.

Dynamic Replica is a synthetic dataset of stereo videos featuring humans and animals in virtual environments. It is a benchmark for dynamic disparity/depth estimation and 3D reconstruction consisting of 145,200 stereo frames (524 videos). The dataset contains annotations for left and right views that include: camera intrinsics and extrinsics, image depth, instance segmentation masks, binary foreground / background segmentation masks, optical flow, long-range pixel trajectories

The High-Quality Wide Multi-Channel Attack database (HQ-WMCA) database consists of 2904 short multi-modal video recordings of both bona-fide and presentation attacks. There are 555 bonafide presentations from 51 participants and the remaining 2349 are presentation attacks. The data is recorded from several channels including color, depth, thermal, infrared (spectra), and short-wave infrared (spectra).

HUMAN4D is a large and multimodal 4D dataset that contains a variety of human activities simultaneously captured by a professional marker-based MoCap, a volumetric capture and an audio recording system. By capturing 2 female and $2$ male professional actors performing various full-body movements and expressions, HUMAN4D provides a diverse set of motions and poses encountered as part of single- and multi-person daily, physical and social activities (jumping, dancing, etc. ), along with multi-RGBD (mRGBD), volumetric and audio data. Description from: HUMAN4D: A Human-Centric Multimodal Dataset for Motions and Immersive Media Image source: https://github.com/tofis/human4d_dataset

Middlebury 2005 is a stereo dataset of indoor scenes. Source: https://vision.middlebury.edu/stereo/data/scenes2005/ Image Source: https://vision.middlebury.edu/stereo/data/scenes2005/

Are current 3D object tracking methods truely robust enough for low-fidelity depth sensors like the iPhone LiDAR? We introduce DTTD-Mobile (fully compatible w/ YCB toolbox), a new benchmark built on real-world data captured from mobile devices; 18 objects observed in 100 videos with 47,668 sampled frames and 114,143 object annotations. We evaluate several popular methods—including BundleSDF, ES6D, MegaPose, and DenseFusion—and highlight their limitations in this challenging setting. Keywords: ObjectPoseEstimation, MobileAI, EdgeAI, ARVR, CVPR

The Oulu-NPU face presentation attack detection database consists of 4950 real access and attack videos. These videos were recorded using the front cameras of six mobile devices (Samsung Galaxy S6 edge, HTC Desire EYE, MEIZU X5, ASUS Zenfone Selfie, Sony XPERIA C5 Ultra Dual and OPPO N3) in three sessions with different illumination conditions and background scenes. The presentation attack types considered in the OULU-NPU database are print and video-replay. The 2D face artefacts were created using two printers and two display devices. The videos of the 55 subjects are divided into three subject-disjoint subsets for training, development and testing. Four test protocols are used to evaluate the generalization capability of face PAD methods across three covariates: unknown environmental conditions (namely illumination and background scene), acquisition devices and presentation attack instruments (PAI). Each of the four unambiguously defined evaluation protocols introduces at least one previously unseen condition to the test set, which enables a fair comparison on the generalization capabilities between new and existing approaches. Image Source: https://www.researchgate.net/profile/Neil-Robertson/publication/333834759/figure/fig5/AS:897964780302339@1591102895306/Samples-from-the-OULU-NPU-database-From-top-to-bottom-is-the-three-sessions-with_W640.jpg

We learn high fidelity human depths by leveraging a collection of social media dance videos scraped from the TikTok mobile social networking application. It is by far one of the most popular video sharing applications across generations, which include short videos (10-15 seconds) of diverse dance challenges as shown above. We manually find more than 300 dance videos that capture a single person performing dance moves from TikTok dance challenge compilations for each month, variety, type of dances, which are moderate movements that do not generate excessive motion blur. For each video, we extract RGB images at 30 frame per second, resulting in more than 100K images. We segmented these images using Removebg application, and computed the UV coordinates from DensePose. Download TikTok Dataset: * Please use the dataset only for the research purpose. * The dataset can be viewed and downloaded from the Kaggle page. (you need to make an account in Kaggle to be able to download the data. It is free!) * The dataset can also be downloaded from here (42 GB). The dataset resolution is: (1080 x 604) * The original YouTube videos corresponding to each sequence and the dance name can be downloaded from here (2.6 GB).

The Stanford Light Field Archive is a collection of several light fields for research in computer graphics and vision. Source: http://lightfield.stanford.edu/ Image Source: http://lightfield.stanford.edu/

TransCG is the first large-scale real-world dataset for transparent object depth completion and grasping, which contains 57,715 RGB-D images of 51 transparent objects and many opaque objects captured from different perspectives (~240 viewpoints) of 130 scenes under real-world settings. The samples are captured by two different types of cameras (Realsense D435 & L515). The following data is provided: - The 3D model of the transparent object; - The 6dpose of the transparent object in each viewpoint of each scene; - The raw RGB-D image, and the ground-truth refined depth image; - The mask of the transparent objects; - The ground-truth surface normals of every sample.

A multimodal database for eye blink detection and attention level estimation. Source: mEBAL: A Multimodal Database for Eye Blink Detection and Attention Level Estimation

Mid-Air, The Montefiore Institute Dataset of Aerial Images and Records, is a multi-purpose synthetic dataset for low altitude drone flights. It provides a large amount of synchronized data corresponding to flight records for multi-modal vision sensors and navigation sensors mounted on board of a flying quadcopter. Multi-modal vision sensors capture RGB pictures, relative surface normal orientation, depth, object semantics and stereo disparity. Source: Mid-Air Dataset

DurLAR is a high-fidelity 128-channel 3D LiDAR dataset with panoramic ambient (near infrared) and reflectivity imagery for multi-modal autonomous driving applications. Compared to existing autonomous driving task datasets, DurLAR has the following novel features: - High vertical resolution LiDAR with 128 channels, which is twice that of any existing datasets, full 360 degree depth, range accuracy to ±2 cm at 20-50m. - Ambient illumination (near infrared) and reflectivity panoramic imagery are made available in the Mono16 format (2048 × 128 resolution), with this being only dataset to make this provision. - No rolling shutter effect, as our flash LiDAR captures all 128 channels simultaneously. - Ambient illumination data is recorded via an on-board lux meter, which is again not available in previous datasets. - High-fidelity GNSS/INS available via an onboard OxTS navigation unit operating at 100 Hz and receiving position and timing data from multiple GNSS con-stellations in addition to GPS. - KITTI data format adopted as the de facto dataset format such that it can be parsed using both the DurLAR development kit and existing KITTI-compatible tools. - Diversity over repeated locations such that the dataset has been collected under diverse environmental and weather conditions over the same driving route with additional variations in the time of day relative to environmental conditions. ## Sensor placement - LiDAR: Ouster OS1-128 LiDAR sensor with 128 channels vertical resolution - Stereo Camera: Carnegie Robotics MultiSense S21 stereo camera with grayscale, colour, and IR enhanced imagers, 2048x1088 @ 2MP resolution - GNSS/INS: OxTS RT3000v3 global navigation satellite and inertial navigation system, supporting localization from GPS, GLONASS, BeiDou, Galileo, PPP and SBAS constellations - Lux Meter: Yocto Light V3, a USB ambient light sensor (lux meter), measuring ambient light up to 100,000 lux

The Helvipad dataset is a real-world stereo dataset designed for omnidirectional depth estimation. It comprises 39,553 paired equirectangular images captured using a top-bottom 360° camera setup and corresponding pixel-wise depth and disparity labels derived from LiDAR point clouds. The dataset spans diverse indoor and outdoor scenes under varying lighting conditions, including night-time environments.

The Middlebury 2006 is a stereo dataset of indoor scenes with multiple handcrafted layouts. Source: https://vision.middlebury.edu/stereo/data/scenes2006/ Image Source: https://vision.middlebury.edu/stereo/data/scenes2006/

We present a large-scale dataset for 3D urban scene understanding. Compared to existing datasets, our dataset consists of 75 outdoor urban scenes with diverse backgrounds, encompassing over 15,000 images. These scenes offer 360◦ hemispherical views, capturing diverse foreground objects illuminated under various lighting conditions. Additionally, our dataset encompasses scenes that are not limited to forward-driving views, addressing the limitations of previous datasets such as limited overlap and coverage between camera views. The closest pre-existing dataset for generalizable evaluation is DTU [2] (80 scenes) which comprises mostly indoor objects and does not provide multiple foreground objects or background scenes. We use the Parallel Domain synthetic data generation to render high-fidelity 360◦ scenes. We select 3 different maps i.e. SF 6thAndMission, SF GrantAndCalifornia and SF VanNessAveAndTurkSt and sample 75 different scenes across all 3 maps as our backgrounds (All 75 scenes across 3 maps are significantly different road scenes from each other, captured at different viewpoints in the city). We select 20 different cars in 50 different textures for training and randomly sample from 1 to 4 cars to render in a scene. We refer to this dataset as NeRDS 360: NeRF for Reconstruction, Decomposition and Scene Synthesis of 360◦ outdoor scenes. In total, we generate 15k renderings by sampling 200 cameras in a hemispherical dome at a fixed distance from the center of cars. We held out 5 scenes with 4 different cars and different backgrounds for testing, comprising 100 cameras distributed uniformly sampled in the upper hemisphere, different from the camera distributions used for training. We use the diverse validation camera distribution to test our approach’s ability to generalize to unseen viewpoints as well as unseen scenes during training. Our dataset and the corresponding task is extremely challenging due to occlusions, diversity of backgrounds, and rendered objects with various lightning and shadows. Our task entails reconstructing 360◦ hemispherical views of complete scenes using a handful of observations i.e. 1 to 5 as shown by red cameras whereas evaluating using all 100 hemispherical views, hence our task requires strong priors for novel view synthesis of outdoor scenes. Tasks our dataset support: - Generaliazable Novel view synthesis (Few shot evaluation) - Novel view synthesis (Overfitting evaluation) - 6D pose estimation - Object editing - Depth estimation - Semantic Segmentation - Instance Segmentation

Endoscopic stereo reconstruction for surgical scenes gives rise to specific problems, including the lack of clear corner features, highly specular surface properties, and the presence of blood and smoke. These issues present difficulties for both stereo reconstruction itself and also for standardised dataset production. We present a stereo-endoscopic reconstruction validation dataset based on cone-beam CT (SERV-CT). Two ex vivo small porcine full torso cadavers were placed within the view of the endoscope with both the endoscope and target anatomy visible in the CT scan. Subsequent orientation of the endoscope was manually aligned to match the stereoscopic view and benchmark disparities, depths and occlusions are calculated. The requirement of a CT scan limited the number of stereo pairs to 8 from each ex vivo sample. For the second sample an RGB surface was acquired to aid alignment of smooth, featureless surfaces. Repeated manual alignments showed an RMS disparity accuracy of around 2 pixels and a depth accuracy of about 2 mm. A simplified reference dataset is provided consisting of endoscope image pairs with corresponding calibration, disparities, depths, and occlusions covering the majority of the endoscopic image and a range of tissue types, including smooth specular surfaces, as well as significant variation of depth. The SERV-CT dataset provides an easy-to-use stereoscopic validation for surgical applications with smooth reference disparities and depths covering the majority of the endoscopic image.

SYNS-Patches dataset, which is a subset of SYNS. The original SYNS is composed of aligned image and LiDAR panoramas from 92 different scenes belonging to a wide variety of environments, such as Agriculture, Natural (e.g. forests and fields), Residential, Industrial and Indoor. It represents the subset of patches from each scene extracted at eye level at 20 degree intervals of a full horizontal rotation. This results in 18 images per scene and a total dataset size of 1656. Source: The Monocular Depth Estimation Challenge Image Source: https://arxiv.org/pdf/2211.12174v1.pdf

Provides a large-scale synthetic dataset which contains accurate ground truth depth of various photo-realistic scenes. Source: 3D Ken Burns Effect from a Single Image

CocoDoom is a collection of pre-recorded data extracted from Doom gaming sessions along with annotations in the MS Coco format. Source: ResearchDoom and CocoDoom: Learning Computer Vision with Games

The DCM dataset is composed of 772 annotated images from 27 golden age comic books. We freely collected them from the free public domain collection of digitized comic books Digital Comics Museum. One album per available publisher was selected to get as many different styles as possible. We made ground-truth bounding boxes of all panels, all characters (body + faces), small or big, human-like or animal-like. Image source: https://gitlab.univ-lr.fr/crigau02/dcm-dataset/-/tree/master

The eBDtheque database is a selection of one hundred comic pages from America, Japan (manga) and Europe. Image source: http://ebdtheque.univ-lr.fr/database/

EDEN (Enclosed garDEN) is a multimodal synthetic dataset, a dataset for nature-oriented applications. The dataset features more than 300K images captured from more than 100 garden models. Each image is annotated with various low/high-level vision modalities, including semantic segmentation, depth, surface normals, intrinsic colors, and optical flow. Source: EDEN: Multimodal Synthetic Dataset of Enclosed GarDEN Scenes Image Source: https://lhoangan.github.io/eden/

The endoscopic SLAM dataset (EndoSLAM) is a dataset for depth estimation approach for endoscopic videos. It consists of both ex-vivo and synthetically generated data. The ex-vivo part of the dataset includes standard as well as capsule endoscopy recordings. The dataset is divided into 35 sub-datasets. Specifically, 18, 5 and 12 sub-datasets exist for colon, small intestine and stomach respectively. Source: https://github.com/CapsuleEndoscope/EndoSLAM Image Source: https://github.com/CapsuleEndoscope/EndoSLAM

The MLFP dataset consists of face presentation attacks captured with seven 3D latex masks and three 2D print attacks. The dataset contains videos captured from color, thermal and infrared channels. Source: Learning One Class Representations for Face Presentation Attack Detection using Multi-channel Convolutional Neural Networks Image Source: http://iab-rubric.org/papers/2017_cvprw_18.pdf