6D Pose Estimation using RGB

The YCB-Video dataset is a large-scale video dataset for 6D object pose estimation. provides accurate 6D poses of 21 objects from the YCB dataset observed in 92 videos with 133,827 frames. Source: https://rse-lab.cs.washington.edu/projects/posecnn/ Image Source: https://www.researchgate.net/figure/Examples-of-refined-poses-on-the-YCB-Video-dataset-which-use-results-from-PoseCNN-Xiang_fig6_339663565

T-LESS is a dataset for estimating the 6D pose, i.e. translation and rotation, of texture-less rigid objects. The dataset features thirty industry-relevant objects with no significant texture and no discriminative color or reflectance properties. The objects exhibit symmetries and mutual similarities in shape and/or size. Compared to other datasets, a unique property is that some of the objects are parts of others. The dataset includes training and test images that were captured with three synchronized sensors, specifically a structured-light and a time-of-flight RGB-D sensor and a high-resolution RGB camera. There are approximately 39K training and 10K test images from each sensor. Additionally, two types of 3D models are provided for each object, i.e. a manually created CAD model and a semi-automatically reconstructed one. Training images depict individual objects against a black background. Test images originate from twenty test scenes having varying complexity, which increases from simple scenes with several isolated objects to very challenging ones with multiple instances of several objects and with a high amount of clutter and occlusion. The images were captured from a systematically sampled view sphere around the object/scene, and are annotated with accurate ground truth 6D poses of all modeled objects. Source: http://cmp.felk.cvut.cz/t-less/ Image Source: http://cmp.felk.cvut.cz/t-less/

The LM (Linemod) dataset is a valuable resource introduced by Stefan Hinterstoisser and colleagues in their research on model-based training, detection, and pose estimation of texture-less 3D objects in heavily cluttered scenes¹. Let's delve into the details: 1. Purpose and Context: - The primary goal of the LM dataset is to facilitate the development and evaluation of methods for detecting and estimating the 6 degrees-of-freedom pose of texture-less 3D objects. - It specifically targets scenarios where objects lack distinctive textures and are embedded in complex backgrounds with occlusions. 2. Methodology: - The dataset builds upon the LINEMOD approach, which combines depth and color information to create templates representing different views of an object. - LINEMOD templates are learned from 3D models and serve as a basis for object detection. - The initial LINEMOD method had limitations, including online template learning and approximate pose estimation. 3. Improvements and Contributions: - Hinterstoisser et al. enhance LINEMOD by incorporating accurate 3D models of objects. - Their approach leverages the 3D model to address the shortcomings of the original LINEMOD. - Notable improvements include better pose estimation and reduced false positives. - The proposed framework is suitable for robotics applications, such as object manipulation. 4. Dataset Details: - The LM dataset consists of 15 registered video sequences, each containing over 1100 frames. - These sequences feature 15 different texture-less household objects. - Objects in the dataset exhibit discriminative color, shape, and size characteristics. - Researchers can use this dataset to evaluate and compare their methods for object detection and pose estimation. LM Dataset Example In summary, the LM dataset provides a valuable benchmark for advancing the field of 6D object pose estimation, especially in scenarios where texture information is limited¹². Researchers can access this dataset to test and refine their algorithms, ultimately contributing to advancements in robotics and machine vision. (1) Model Based Training, Detection and Pose ... - Stefan HINTERSTOISSER. http://stefan-hinterstoisser.com/papers/hinterstoisser2012accv.pdf. (2) Datasets - BOP: Benchmark for 6D Object Pose Estimation. https://bop.felk.cvut.cz/datasets/. (3) paroj/linemod_dataset: Hinterstoisser et al. ACCV12 dataset - GitHub. https://github.com/paroj/linemod_dataset.

ApolloCar3DT is a dataset that contains 5,277 driving images and over 60K car instances, where each car is fitted with an industry-grade 3D CAD model with absolute model size and semantically labelled keypoints. This dataset is above 20 times larger than PASCAL3D+ and KITTI, the current state-of-the-art. Source: ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Driving Image Source: http://apolloscape.auto/car_instance.html

The Fraunhofer IPA Bin-Picking dataset is a large-scale dataset comprising both simulated and real-world scenes for various objects (potentially having symmetries) and is fully annotated with 6D poses. A pyhsics simulation is used to create scenes of many parts in bulk by dropping objects in a random position and orientation above a bin. Additionally, this dataset extends the Siléane dataset by providing more samples. This allows to e.g. train deep neural networks and benchmark the performance on the public Siléane dataset Source: https://www.bin-picking.ai/en/dataset.html Image Source: https://arxiv.org/abs/1912.12125

Estimating camera motion in deformable scenes poses a complex and open research challenge. Most existing non-rigid structure from motion techniques assume to observe also static scene parts besides deforming scene parts in order to establish an anchoring reference. However, this assumption does not hold true in certain relevant application cases such as endoscopies. To tackle this issue with a common benchmark, we introduce the Drunkard’s Dataset, a challenging collection of synthetic data targeting visual navigation and reconstruction in deformable environments. This dataset is the first large set of exploratory camera trajectories with ground truth inside 3D scenes where every surface exhibits non-rigid deformations over time. Simulations in realistic 3D buildings lets us obtain a vast amount of data and ground truth labels, including camera poses, RGB images and depth, optical flow and normal maps at high resolution and quality.

UW Indoor Scenes (UW-IS) Occluded dataset is curated using commodity hardware (Intel RealSense D435) to reflect real world robotics scenarios. It consists of two completely different indoor environments. The first environment is a lounge where the objects are placed on a tabletop. The second environment is a mock warehouse setup where the objects are placed on a shelf. For each of these environments, we have RGB-D images from 36 videos comprising five to seven objects each, taken from distances up to approximately 2m. The videos cover two different lighting conditions, three different levels of object separation for three different object categories (i.e., kitchen objects, food items, and tools/miscellaneous). The first level of object separation is such that there is no object occlusion. The second level of object separation is such that some occlusion occurs, while the third level is where the objects are placed extremely close together. Overall, the dataset considers 20 object classes and consists of 8,456 images, which have a total of 42,902 object instances. We also provide instance segmentation masks and 6D pose annotations for all the images generated using LabelFusion (Marion et al., 2018)