6D Pose Estimation using RGBD

The YCB-Video dataset is a large-scale video dataset for 6D object pose estimation. provides accurate 6D poses of 21 objects from the YCB dataset observed in 92 videos with 133,827 frames. Source: https://rse-lab.cs.washington.edu/projects/posecnn/ Image Source: https://www.researchgate.net/figure/Examples-of-refined-poses-on-the-YCB-Video-dataset-which-use-results-from-PoseCNN-Xiang_fig6_339663565

T-LESS is a dataset for estimating the 6D pose, i.e. translation and rotation, of texture-less rigid objects. The dataset features thirty industry-relevant objects with no significant texture and no discriminative color or reflectance properties. The objects exhibit symmetries and mutual similarities in shape and/or size. Compared to other datasets, a unique property is that some of the objects are parts of others. The dataset includes training and test images that were captured with three synchronized sensors, specifically a structured-light and a time-of-flight RGB-D sensor and a high-resolution RGB camera. There are approximately 39K training and 10K test images from each sensor. Additionally, two types of 3D models are provided for each object, i.e. a manually created CAD model and a semi-automatically reconstructed one. Training images depict individual objects against a black background. Test images originate from twenty test scenes having varying complexity, which increases from simple scenes with several isolated objects to very challenging ones with multiple instances of several objects and with a high amount of clutter and occlusion. The images were captured from a systematically sampled view sphere around the object/scene, and are annotated with accurate ground truth 6D poses of all modeled objects. Source: http://cmp.felk.cvut.cz/t-less/ Image Source: http://cmp.felk.cvut.cz/t-less/

REAL275 is a benchmark for category-level pose estimation. It contains 4300 training frames, 950 validation and 2750 for testing across 18 different real scenes.

The LM (Linemod) dataset is a valuable resource introduced by Stefan Hinterstoisser and colleagues in their research on model-based training, detection, and pose estimation of texture-less 3D objects in heavily cluttered scenes¹. Let's delve into the details: 1. Purpose and Context: - The primary goal of the LM dataset is to facilitate the development and evaluation of methods for detecting and estimating the 6 degrees-of-freedom pose of texture-less 3D objects. - It specifically targets scenarios where objects lack distinctive textures and are embedded in complex backgrounds with occlusions. 2. Methodology: - The dataset builds upon the LINEMOD approach, which combines depth and color information to create templates representing different views of an object. - LINEMOD templates are learned from 3D models and serve as a basis for object detection. - The initial LINEMOD method had limitations, including online template learning and approximate pose estimation. 3. Improvements and Contributions: - Hinterstoisser et al. enhance LINEMOD by incorporating accurate 3D models of objects. - Their approach leverages the 3D model to address the shortcomings of the original LINEMOD. - Notable improvements include better pose estimation and reduced false positives. - The proposed framework is suitable for robotics applications, such as object manipulation. 4. Dataset Details: - The LM dataset consists of 15 registered video sequences, each containing over 1100 frames. - These sequences feature 15 different texture-less household objects. - Objects in the dataset exhibit discriminative color, shape, and size characteristics. - Researchers can use this dataset to evaluate and compare their methods for object detection and pose estimation. LM Dataset Example In summary, the LM dataset provides a valuable benchmark for advancing the field of 6D object pose estimation, especially in scenarios where texture information is limited¹². Researchers can access this dataset to test and refine their algorithms, ultimately contributing to advancements in robotics and machine vision. (1) Model Based Training, Detection and Pose ... - Stefan HINTERSTOISSER. http://stefan-hinterstoisser.com/papers/hinterstoisser2012accv.pdf. (2) Datasets - BOP: Benchmark for 6D Object Pose Estimation. https://bop.felk.cvut.cz/datasets/. (3) paroj/linemod_dataset: Hinterstoisser et al. ACCV12 dataset - GitHub. https://github.com/paroj/linemod_dataset.

Estimating camera motion in deformable scenes poses a complex and open research challenge. Most existing non-rigid structure from motion techniques assume to observe also static scene parts besides deforming scene parts in order to establish an anchoring reference. However, this assumption does not hold true in certain relevant application cases such as endoscopies. To tackle this issue with a common benchmark, we introduce the Drunkard’s Dataset, a challenging collection of synthetic data targeting visual navigation and reconstruction in deformable environments. This dataset is the first large set of exploratory camera trajectories with ground truth inside 3D scenes where every surface exhibits non-rigid deformations over time. Simulations in realistic 3D buildings lets us obtain a vast amount of data and ground truth labels, including camera poses, RGB images and depth, optical flow and normal maps at high resolution and quality.

This dataset comprehends the 3D building information model (in IFC and Revit formats), manually elaborated based on the terrestrial laser scanner of the sequence 2 of ConSLAM, and the refined ground truth (GT) poses (in TUM format) of sessions 2, 3, 4, and 5 of the open-access ConSLAM dataset (which provides camera, LiDAR, and IMU measurements). This dataset enables the evaluation of pose estimation algorithms, such as localization, odometry, SLAM, and Bundle adjustment. Given the differences between the real-world and the 3D reference BIM model (Scan-Map deviations), the dataset is appropriate for evaluating long-term localization and mapping capabilities, i.e., methods that are robust to changes in the environment. In particular, in a real-world construction environment with normal clutter (not as in the Hilti dataset).

UW Indoor Scenes (UW-IS) Occluded dataset is curated using commodity hardware (Intel RealSense D435) to reflect real world robotics scenarios. It consists of two completely different indoor environments. The first environment is a lounge where the objects are placed on a tabletop. The second environment is a mock warehouse setup where the objects are placed on a shelf. For each of these environments, we have RGB-D images from 36 videos comprising five to seven objects each, taken from distances up to approximately 2m. The videos cover two different lighting conditions, three different levels of object separation for three different object categories (i.e., kitchen objects, food items, and tools/miscellaneous). The first level of object separation is such that there is no object occlusion. The second level of object separation is such that some occlusion occurs, while the third level is where the objects are placed extremely close together. Overall, the dataset considers 20 object classes and consists of 8,456 images, which have a total of 42,902 object instances. We also provide instance segmentation masks and 6D pose annotations for all the images generated using LabelFusion (Marion et al., 2018)

The dataset includes the synthetic data generated from rendering the 3D meshes of LM objects and several household objects in Blender for training 6D pose estimation algorithms. The whole dataset contains synthetic data for 18 objects (13 from LM and 5 from household objects), with 20,000 data samples for each object. Each data sample includes an RGB image in .png format and a depth image in .exr format. Each sample has the annotations of mask labels in .png format and the ground truth pose labels saved in .json files. Apart from the training data, the 3D meshes of the objects and the pre-trained models of the 6D pose estimation algorithm are also included. The whole dataset takes approximately ~1T of storage memory.

The YCB-Ev dataset contains synchronized RGB-D frames and event data that enables evaluating 6DoF object pose estimation algorithms using these modalities. This dataset provides ground truth 6DoF object poses for the same 21 YCB objects that were used in the YCB-Video (YCB-V) dataset, allowing for cross-dataset algorithm performance evaluation. The dataset consists of 21 synchronized event and RGB-D sequences, totalling 13,851 frames (7 minutes and 43 seconds of event data). Notably, 12 of these sequences feature the same object arrangement as the YCB-V subset used in the BOP challenge. Ground truth poses are generated by detecting objects in the RGB-D frames, interpolating the poses to align with the event timestamps, and then transferring them to the event coordinate frame using extrinsic calibration.