Visual Odometry

KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute) is one of the most popular datasets for use in mobile robotics and autonomous driving. It consists of hours of traffic scenarios recorded with a variety of sensor modalities, including high-resolution RGB, grayscale stereo cameras, and a 3D laser scanner. Despite its popularity, the dataset itself does not contain ground truth for semantic segmentation. However, various researchers have manually annotated parts of the dataset to fit their necessities. Álvarez et al. generated ground truth for 323 images from the road detection challenge with three classes: road, vertical, and sky. Zhang et al. annotated 252 (140 for training and 112 for testing) acquisitions – RGB and Velodyne scans – from the tracking challenge for ten object categories: building, sky, road, vegetation, sidewalk, car, pedestrian, cyclist, sign/pole, and fence. Ros et al. labeled 170 training images and 46 testing images (from the visual odometry challenge) with 11 classes: building, tree, sky, car, sign, road, pedestrian, fence, pole, sidewalk, and bicyclist. Source: A Review on Deep Learning Techniques Applied to Semantic Segmentation Image Source: http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d

TUM RGB-D is an RGB-D dataset. It contains the color and depth images of a Microsoft Kinect sensor along the ground-truth trajectory of the sensor. The data was recorded at full frame rate (30 Hz) and sensor resolution (640x480). The ground-truth trajectory was obtained from a high-accuracy motion-capture system with eight high-speed tracking cameras (100 Hz). Source: https://vision.in.tum.de/data/datasets/rgbd-dataset Image Source: https://vision.in.tum.de/research/rgb-d_sensors_kinect

The Replay-Attack Database for face spoofing consists of 1300 video clips of photo and video attack attempts to 50 clients, under different lighting conditions. All videos are generated by either having a (real) client trying to access a laptop through a built-in webcam or by displaying a photo or a video recording of the same client for at least 9 seconds. Source: https://www.idiap.ch/dataset/replayattack Image Source: https://www.researchgate.net/figure/Sample-images-from-the-PRINT-ATTACK-1-and-REPLAY-ATTACK-12-databases-Top-and-bottom_fig1_330400888

Virtual KITTI is a photo-realistic synthetic video dataset designed to learn and evaluate computer vision models for several video understanding tasks: object detection and multi-object tracking, scene-level and instance-level semantic segmentation, optical flow, and depth estimation. Virtual KITTI contains 50 high-resolution monocular videos (21,260 frames) generated from five different virtual worlds in urban settings under different imaging and weather conditions. These worlds were created using the Unity game engine and a novel real-to-virtual cloning method. These photo-realistic synthetic videos are automatically, exactly, and fully annotated for 2D and 3D multi-object tracking and at the pixel level with category, instance, flow, and depth labels (cf. below for download links). Source: https://europe.naverlabs.com/research/computer-vision/proxy-virtual-worlds-vkitti-1/ Image Source: https://arxiv.org/pdf/1605.06457.pdf

A dataset for robot navigation task and more. The data is collected in photo-realistic simulation environments in the presence of various light conditions, weather and moving objects. Source: TartanAir: A Dataset to Push the Limits of Visual SLAM

The MSU-MFSD dataset contains 280 video recordings of genuine and attack faces. 35 individuals have participated in the development of this database with a total of 280 videos. Two kinds of cameras with different resolutions (720×480 and 640×480) were used to record the videos from the 35 individuals. For the real accesses, each individual has two video recordings captured with the Laptop cameras and Android, respectively. For the video attacks, two types of cameras, the iPhone and Canon cameras were used to capture high definition videos on each of the subject. The videos taken with Canon camera were then replayed on iPad Air screen to generate the HD replay attacks while the videos recorded by the iPhone mobile were replayed itself to generate the mobile replay attacks. Photo attacks were produced by printing the 35 subjects’ photos on A3 papers using HP colour printer. The recording videos with respect to the 35 individuals were divided into training (15 subjects with 120 videos) and testing (40 subjects with 160 videos) datasets, respectively. Source: Enhance the Motion Cues for Face Anti-Spoofing using CNN-LSTM Architecture Image Source: face anti-spoofing based on color texture analysis

CASIA-MFSD is a dataset for face anti-spoofing. It contains 50 subjects, and 12 videos for each subject under different resolutions and light conditions. Three different spoof attacks are designed: replay, warp print and cut print attacks. The database contains 600 video recordings, in which 240 videos of 20 subjects are used for training and 360 videos of 30 subjects for testing. Source: Improving Face Anti-Spoofing by 3D Virtual Synthesis Image Source: https://link.springer.com/referenceworkentry/10.1007%2F978-1-4899-7488-4_9067

Virtual KITTI 2 is an updated version of the well-known Virtual KITTI dataset which consists of 5 sequence clones from the KITTI tracking benchmark. In addition, the dataset provides different variants of these sequences such as modified weather conditions (e.g. fog, rain) or modified camera configurations (e.g. rotated by 15◦). For each sequence we provide multiple sets of images containing RGB, depth, class segmentation, instance segmentation, flow, and scene flow data. Camera parameters and poses as well as vehicle locations are available as well. In order to showcase some of the dataset’s capabilities, we ran multiple relevant experiments using state-of-the-art algorithms from the field of autonomous driving. The dataset is available for download at https://europe.naverlabs.com/Research/Computer-Vision/Proxy-Virtual-Worlds.

The Event-Camera Dataset is a collection of datasets with an event-based camera for high-speed robotics. The data also include intensity images, inertial measurements, and ground truth from a motion-capture system. An event-based camera is a revolutionary vision sensor with three key advantages: a measurement rate that is almost 1 million times faster than standard cameras, a latency of 1 microsecond, and a high dynamic range of 130 decibels (standard cameras only have 60 dB). These properties enable the design of a new class of algorithms for high-speed robotics, where standard cameras suffer from motion blur and high latency. All the data are released both as text files and binary (i.e., rosbag) files. Source: The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM

SiW provides live and spoof videos from 165 subjects. For each subject, we have 8 live and up to 20 spoof videos, in total 4,478 videos. All videos are in 30 fps, about 15 second length, and 1080P HD resolution. The live videos are collected in four sessions with variations of distance, pose, illumination and expression. The spoof videos are collected with several attacks such as printed paper and replay.

CASIA-FASD is a small face anti-spoofing dataset containing 50 subjects. Source: Learning Generalizable and Identity-Discriminative Representations for Face Anti-Spoofing Image Source: https://arxiv.org/abs/1511.06316

CelebA-Spoof is a large-scale face anti-spoofing dataset with the following properties: 1. Quantity: CelebA-Spoof comprises of 625,537 pictures of 10,177 subjects, significantly larger than the existing datasets. 2. Diversity: The spoof images are captured from 8 scenes (2 environments * 4 illumination conditions) with more than 10 sensors. 3. Annotation Richness: CelebA-Spoof contains 10 spoof type annotations, as well as the 40 attribute annotations inherited from the original CelebA dataset. Source: CelebA-Spoof: Large-Scale Face Anti-Spoofing Dataset with Rich Annotations

Dataset for face anti-spoofing in terms of both subjects and modalities. Specifically, it consists of subjects with videos and each sample has modalities (i.e., RGB, Depth and IR). Source: CASIA-SURF: A Large-scale Multi-modal Benchmark for Face Anti-spoofing

The Replay-Mobile Database for face spoofing consists of 1190 video clips of photo and video attack attempts to 40 clients, under different lighting conditions. These videos were recorded with current devices from the market -- an iPad Mini2 (running iOS) and a LG-G4 smartphone (running Android). This Database was produced at the Idiap Research Institute (Switzerland) within the framework of collaboration with Galician Research and Development Center in Advanced Telecommunications - Gradiant (Spain). Source: Replay-Mobile Image Source: https://core.ac.uk/download/pdf/148024307.pdf

The New College Data is a freely available dataset collected from a robot completing several loops outdoors around the New College campus in Oxford. The data includes odometry, laser scan, and visual information. The dataset URL is not working anymore. Source: https://www.ros.org/news/2010/07/new-college-dataset-parser-for-ros.html

4Seasons is adataset covering seasonal and challenging perceptual conditions for autonomous driving. Source: 4Seasons: A Cross-Season Dataset for Multi-Weather SLAM in Autonomous Driving Image Source: Wenzel et al

HiFiMask is a large-scale High-Fidelity Mask dataset, namely CASIA-SURF HiFiMask (briefly HiFiMask). It contains a total amount of 54,600 videos are recorded from 75 subjects with 225 realistic masks by 7 new kinds of sensors.

EuRoC MAV is a visual-inertial datasets collected on-board a Micro Aerial Vehicle (MAV). The dataset contains stereo images, synchronized IMU measurements, and accurate motion and structure ground-truth. The datasets facilitates the design and evaluation of visual-inertial localization algorithms on real flight data Source: https://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets Image Source: https://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets

TUM monoVO is a dataset for evaluating the tracking accuracy of monocular Visual Odometry (VO) and SLAM methods. It contains 50 real-world sequences comprising over 100 minutes of video, recorded across different environments – ranging from narrow indoor corridors to wide outdoor scenes. All sequences contain mostly exploring camera motion, starting and ending at the same position: this allows to evaluate tracking accuracy via the accumulated drift from start to end, without requiring ground-truth for the full sequence. In contrast to existing datasets, all sequences are photometrically calibrated: the dataset creators provide the exposure times for each frame as reported by the sensor, the camera response function and the lens attenuation factors (vignetting). Source: https://vision.in.tum.de/data/datasets/mono-dataset Image Source: https://vision.in.tum.de/data/datasets/mono-dataset

The High-Quality Wide Multi-Channel Attack database (HQ-WMCA) database consists of 2904 short multi-modal video recordings of both bona-fide and presentation attacks. There are 555 bonafide presentations from 51 participants and the remaining 2349 are presentation attacks. The data is recorded from several channels including color, depth, thermal, infrared (spectra), and short-wave infrared (spectra).

The Oulu-NPU face presentation attack detection database consists of 4950 real access and attack videos. These videos were recorded using the front cameras of six mobile devices (Samsung Galaxy S6 edge, HTC Desire EYE, MEIZU X5, ASUS Zenfone Selfie, Sony XPERIA C5 Ultra Dual and OPPO N3) in three sessions with different illumination conditions and background scenes. The presentation attack types considered in the OULU-NPU database are print and video-replay. The 2D face artefacts were created using two printers and two display devices. The videos of the 55 subjects are divided into three subject-disjoint subsets for training, development and testing. Four test protocols are used to evaluate the generalization capability of face PAD methods across three covariates: unknown environmental conditions (namely illumination and background scene), acquisition devices and presentation attack instruments (PAI). Each of the four unambiguously defined evaluation protocols introduces at least one previously unseen condition to the test set, which enables a fair comparison on the generalization capabilities between new and existing approaches. Image Source: https://www.researchgate.net/profile/Neil-Robertson/publication/333834759/figure/fig5/AS:897964780302339@1591102895306/Samples-from-the-OULU-NPU-database-From-top-to-bottom-is-the-three-sessions-with_W640.jpg

A multimodal database for eye blink detection and attention level estimation. Source: mEBAL: A Multimodal Database for Eye Blink Detection and Attention Level Estimation

ALTO is a vision-focused dataset for the development and benchmarking of Visual Place Recognition and Localization methods for Unmanned Aerial Vehicles. The dataset is composed of two long (approximately 150km and 260km) trajectories flown by a helicopter over Ohio and Pennsylvania, and it includes high precision GPS-INS ground truth location data, high precision accelerometer readings, laser altimeter readings, and RGB downward facing camera imagery.The dataset also comes with reference imagery over the flight paths, which makes this dataset suitable for VPR benchmarking and other tasks common in Localization, such as image registration and visual odometry.

The endoscopic SLAM dataset (EndoSLAM) is a dataset for depth estimation approach for endoscopic videos. It consists of both ex-vivo and synthetically generated data. The ex-vivo part of the dataset includes standard as well as capsule endoscopy recordings. The dataset is divided into 35 sub-datasets. Specifically, 18, 5 and 12 sub-datasets exist for colon, small intestine and stomach respectively. Source: https://github.com/CapsuleEndoscope/EndoSLAM Image Source: https://github.com/CapsuleEndoscope/EndoSLAM

The MLFP dataset consists of face presentation attacks captured with seven 3D latex masks and three 2D print attacks. The dataset contains videos captured from color, thermal and infrared channels. Source: Learning One Class Representations for Face Presentation Attack Detection using Multi-channel Convolutional Neural Networks Image Source: http://iab-rubric.org/papers/2017_cvprw_18.pdf

CelebA-Spoof is a large-scale face anti-spoofing dataset recently introduced in [53]. The dataset contains 625,537 images of 10,177 celebrities captured under different spoof mediums, environments and illumination conditions. The original dataset proposes three different evaluation protocols. For our experimentation, we focus on the most general ”intra” protocol, in which different spoof types, environments and illumination conditions are used for both training and testing. To generate CelebA-Spoof-Enroll, the personalized version of the CelebA-Spoof anti-spoofing dataset, we start by setting the desired enrollment set size N. We decide for a constant number of enrollment images per user to be consistent with the implementation in typical commercial applications and to simplify the dataset definition. For both the training and test split, we count the number of live samples per user and discard those users having ≤ N live samples. So, if we desire a higher value of N, a larger number of users would get rejected and hence the number of training samples would be less. Note that it is not possible to include all original data, as a number of users in CelebA-Spoof are missing live samples. Nevertheless, only a very small percentage of training and test data is discarded through this process when choosing N < 10. For each accepted user, the first N live samples (ordered according to the ascending alphanumeric ordering of the original filenames) are chosen to define its enrollment set. The rest of the live samples and the spoof samples are marked as query samples. It is important to note that using this deterministic method of obtaining enrollment samples does not introduce unwanted bias as most of the CelebASpoof images are randomly crawled from the internet and are not ordered according to specific criteria. With this setting, we associate a user identifier with each query sample such that queries can be easily mapped to the correct enrollment set. This method of filtering users for a desired enrollment size N is performed for both training and test split. In the rest of the paper, we refer to CelebA-Spoof-EnrollN or in short CASp-EnrollN to describe the personalized version of the dataset with N enrollment images (See paper for additional details)

SiW (Spoofing in the Wild) is a face anti-spoofing dataset recently introduced in [29] where images are extracted from short videos captured at high resolution and 30 frames per second. In total, 4,478 videos are collected from 165 subjects including variations in spoof type, recording device, illumination condition, pose and facial expression. To define train and test sets, we start by following the splitting system described as Protocol 1 in [29]. We further subsample train and test sets by extracting 1 every 10 frames (or every 0.33 seconds) in each video, since consecutive frames are almost identical. Finally, since we observe that simple models can reach very high accuracy, we further make the task harder by creating 2 separate training and evaluation folds in a way that the model is trained and tested on different spoof types. Since the original dataset is collected using two different capturing devices, we can use this information for the definition of enrollment sets. Indeed, in a real scenario, the subject would have to enroll twice when using two different devices, meaning that a unique enrollment set will be generated for each combination of subject and capturing devices. Capturing the sensor bias in the enrollment can be particularly beneficial, for example to detect if the resolution of a face changes, which can indicate a replay attack. To extract enrollment data we choose a single video from each subject and sensor among the available ones. We arbitrarily pick the video without illumination changes and with variation in subject pose, as it resembles the enrollment conditions required in real devices. We then equidistantly sample N frames over the video to construct the enrollment set. As shown in Fig. 2, this allows capturing different poses and facial expressions from the subject. Finally, we exclude all the frames in this video from training and test data to avoid in341 formation leakage and match all the remaining queries from the same subject and sensor to their unique enrollment set. (See paper for additional details)

A novel dataset with a diverse set of sequences in different scenes for evaluating VI odometry. It provides camera images with 1024x1024 resolution at 20 Hz, high dynamic range and photometric calibration. Source: The TUM VI Benchmark for Evaluating Visual-Inertial Odometry

This dataset presents a vision and perception research dataset collected in Rome, featuring RGB data, 3D point clouds, IMU, and GPS data. We introduce a new benchmark targeting visual odometry and SLAM, to advance the research in autonomous robotics and computer vision. This work complements existing datasets by simultaneously addressing several issues, such as environment diversity, motion patterns, and sensor frequency. It uses up-to-date devices and presents effective procedures to accurately calibrate the intrinsic and extrinsic of the sensors while addressing temporal synchronization. During recording, we cover multi-floor buildings, gardens, urban and highway scenarios. Combining handheld and car-based data collections, our setup can simulate any robot (quadrupeds, quadrotors, autonomous vehicles). The dataset includes an accurate 6-dof ground truth based on a novel methodology that refines the RTK-GPS estimate with LiDAR point clouds through Bundle Adjustment. All sequences divided in training and testing are accessible through our website.

A new underwater dataset that has been recorded in an harbor and provides several sequences with synchronized measurements from a monocular camera, a MEMS-IMU and a pressure sensor. Source: The Aqualoc Dataset: Towards Real-Time Underwater Localization from a Visual-Inertial-Pressure Acquisition System

BASEPROD provides comprehensive rover sensor data collected over a 1.7 km traverse, accompanied by high-resolution 2D and 3D drone maps of the terrain. The dataset also includes laser-induced breakdown spectroscopy (LIBS) measurements from key sampling sites along the rover's path, as well as weather station data to contextualize environmental conditions. The rover is a half-scale model of the ExoMars rover, featuring a sophisticated six-wheel triple-bogie suspension system. Each wheel assembly includes three actuators— one for wheel deployment (see e.g., wheel-walking), one for steering, and one for driving — and is equipped with a force-torque sensor. The sensor suite also comprises an RGB-D camera, a thermal camera, and a stereo camera mounted on a mast with pan-tilt capability. Additionally, the dataset contains IMU, GNSS, and gyroscope measurements, facilitating comprehensive analysis. Potential Applications: Research in navigation, localization, mapping, odometry, and terrain classification or segmentation tasks. It provides a rich resource for developing and testing algorithms in planetary exploration conditions.

Estimating camera motion in deformable scenes poses a complex and open research challenge. Most existing non-rigid structure from motion techniques assume to observe also static scene parts besides deforming scene parts in order to establish an anchoring reference. However, this assumption does not hold true in certain relevant application cases such as endoscopies. To tackle this issue with a common benchmark, we introduce the Drunkard’s Dataset, a challenging collection of synthetic data targeting visual navigation and reconstruction in deformable environments. This dataset is the first large set of exploratory camera trajectories with ground truth inside 3D scenes where every surface exhibits non-rigid deformations over time. Simulations in realistic 3D buildings lets us obtain a vast amount of data and ground truth labels, including camera poses, RGB images and depth, optical flow and normal maps at high resolution and quality.

This dataset comprehends the 3D building information model (in IFC and Revit formats), manually elaborated based on the terrestrial laser scanner of the sequence 2 of ConSLAM, and the refined ground truth (GT) poses (in TUM format) of sessions 2, 3, 4, and 5 of the open-access ConSLAM dataset (which provides camera, LiDAR, and IMU measurements). This dataset enables the evaluation of pose estimation algorithms, such as localization, odometry, SLAM, and Bundle adjustment. Given the differences between the real-world and the 3D reference BIM model (Scan-Map deviations), the dataset is appropriate for evaluating long-term localization and mapping capabilities, i.e., methods that are robust to changes in the environment. In particular, in a real-world construction environment with normal clutter (not as in the Hilti dataset).

SuHiFiMask (Surveillance High-Fidelity Mask) extends FAS to real surveillance scenes rather than mimicking low-resolution images and surveillance environments. It contains 10,195 videos from 101 subjects of different age groups, which are collected by 7 mainstream surveillance cameras. Source: Surveillance Face Presentation Attack Detection Challenge

Comprised of real human and wax figure images and videos that endorse the problem of face spoofing detection. The dataset consists of more than 1800 face images and 110 videos of 55 people/waxworks, arranged in training, validation and test sets with a large range in expression, illumination and pose variations. Source: The SWAX Benchmark: Attacking Biometric Systems with Wax Figures

AUT-VI is a super-challenging visual inertial dataset with 126 diverse sequences in 17 locations. This dataset contains dynamic objects, challenging loop-closure/map-reuse, different lighting conditions, reflections, and sudden camera movements to cover all extreme navigation scenarios. Moreover, the Android application for data capture is released to the public to support ongoing development efforts. This dataset aims to exploit the remaining challenges in VIO algorithms, in the hope of improving them to facilitate navigation for visually impaired individuals in both indoor and outdoor settings.