GRAB is a dataset of full-body motions interacting and grasping 3D objects. It contains accurate finger and facial motions as well as the contact between the objects and body. It contains 5 male and 5 female participants and 4 different motion intents. The GRAB dataset also contains binary contact maps between the body and objects. Source: https://github.com/otaheri/GRAB Image Source: https://github.com/otaheri/GRAB

The Evolved Grasping Analysis Dataset (EGAD) comprises over 2000 generated objects aimed at training and evaluating robotic visual grasp detection algorithms. The objects in EGAD are geometrically diverse, filling a space ranging from simple to complex shapes and from easy to difficult to grasp, compared to other datasets for robotic grasping, which may be limited in size or contain only a small number of object classes.

Dex-Net 2.0 is a dataset associating 6.7 million point clouds and analytic grasp quality metrics with parallel-jaw grasps planned using robust quasi-static GWS analysis on a dataset of 1,500 3D object models.

GraspNet-1Billion provides large-scale training data and a standard evaluation platform for the task of general robotic grasping. The dataset contains 97,280 RGB-D image with over one billion grasp poses.

An open database for sharing robotic experience, which provides an initial pool of 15 million video frames, from 7 different robot platforms, and study how it can be used to learn generalizable models for vision-based robotic manipulation. Source: RoboNet: Large-Scale Multi-Robot Learning

ContactPose is a dataset of hand-object contact paired with hand pose, object pose, and RGB-D images. ContactPose has 2306 unique grasps of 25 household objects grasped with 2 functional intents by 50 participants, and more than 2.9 M RGB-D grasp images. Source: ContactPose: A Dataset of Grasps with Object Contact and Hand Pose

ContactDB is a dataset of contact maps for household objects that captures the rich hand-object contact that occurs during grasping, enabled by use of a thermal camera. ContactDB includes 3,750 3D meshes of 50 household objects textured with contact maps and 375K frames of synchronized RGB-D+thermal images. Source: https://arxiv.org/abs/1904.06830 Image Source: https://github.com/samarth-robo/contactdb_utils

Are current 3D object tracking methods truely robust enough for low-fidelity depth sensors like the iPhone LiDAR? We introduce DTTD-Mobile (fully compatible w/ YCB toolbox), a new benchmark built on real-world data captured from mobile devices; 18 objects observed in 100 videos with 47,668 sampled frames and 114,143 object annotations. We evaluate several popular methods—including BundleSDF, ES6D, MegaPose, and DenseFusion—and highlight their limitations in this challenging setting. Keywords: ObjectPoseEstimation, MobileAI, EdgeAI, ARVR, CVPR

A Modular Simulation Framework and Benchmark for Robot Learning.

We leverage knowledge from foundation models to introduce Grasp-Anything, a new large-scale dataset with 1M (one million) samples and 3M objects, substantially surpassing prior datasets in diversity and magnitude. In addition, Grasp-Anything can universally cover objects in our daily lives and offer a great range of object diversity.

Robot grasping is often formulated as a learning problem. With the increasing speed and quality of physics simulations, generating large-scale grasping data sets that feed learning algorithms is becoming more and more popular. An often overlooked question is how to generate the grasps that make up these data sets. In this paper, we review, classify, and compare different grasp sampling strategies. Our evaluation is based on a fine-grained discretization of SE(3) and uses physics-based simulation to evaluate the quality and robustness of the corresponding parallel-jaw grasps. Specifically, we consider more than 1 billion grasps for each of the 21 objects from the YCB data set. This dense data set lets us evaluate existing sampling schemes w.r.t. their bias and efficiency. Our experiments show that some popular sampling schemes contain significant bias and do not cover all possible ways an object can be grasped.

GAS (Grasp Area Segmentation) dataset consists of 10089 RGB images of cluttered scenes grouped into 1121 grasp-area segmentation tasks. For each RGB image we provide a binary segmentation map with the graspable and non-graspable regions for every object in the scene. The dataset can be used for meta-training part-based grasp area estimation networks. For creating the GAS dataset we use the RGB images and corresponding ground truth segmentation masks from the GraspNet 1-Billion dataset.

The Calandra dataset provides the data from a pair of tactile sensors attached to a jaw gripper (left and right) alongside the RGB images. A triplet of samples was captured ’before’, ’during’, and ’after’ grasping a plethora of objects. The objective is to determine the success or the failure of the grasp attempt.

A dataset for position-constrained robot grasp planning.

Event-Stream Dataset is a robotic grasping dataset with 91 objects. Source: Event-based Robotic Grasping Detection with Neuromorphic Vision Sensor and Event-Stream Dataset

Robotic grasp dataset for multi-object multi-grasp evaluation with RGB-D data. This dataset is annotated using the same protocol as Cornell Dataset, and can be used as multi-object extension of Cornell Dataset. Source: https://github.com/ivalab/grasp_multiObject Image Source: https://github.com/ivalab/grasp_multiObject

GraspClutter6D is a large-scale real-world dataset for robust object perception and robotic grasping in cluttered environments. It features 1,000 highly cluttered scenes with dense arrangements (average 14.1 objects/scene with 62.6% occlusion), 200 household, industrial, and warehouse objects captured in 75 diverse environment configurations (bins, shelves, and tables), multi-view data from 4 RGB-D cameras (RealSense D415, D435, Azure Kinect, and Zivid One+), and comprehensive annotations including 736K 6D object poses and 9.3 billion feasible robotic grasps for 52K RGB-D images. The dataset provides a challenging testbed for segmentation, 6D pose estimation, and grasp detection algorithms in realistic cluttered scenarios.

Accompanying expert data and trained models for 2021 IROS paper on Multiview Manipulation.

Introduction NBMOD is a dataset created for researching the task of specific object grasp detection by robots in noisy environments. The dataset comprises three subsets: Simple background Single-object Subset (SSS), Noisy background Single-object Subset (NSS), and Multi-Object grasp detection Subset (MOS). The SSS subset contains 13,500 images, the NSS subset contains 13,000 images, and the MOS subset contains 5,000 images. ## What are the differences of NBMOD? Unlike the renowned Cornell dataset, the NBMOD dataset differs in that its backgrounds are no longer simple whiteboards. The NSS and MOS subsets comprise a substantial number of images with noise, where this noise corresponds to interfering objects unrelated to the target objects for grasping detection. Moreover, in the MOS subset, each image encompasses multiple target objects for grasp detection, which closely resembles real-world working environments.

Includes several sets of synthetic stereo images labelled with grasp rectangles representing parallel-jaw grasps (Cornell-like format). The set was introduced in the range of the ICAR paper "Automatic generation of realistic training data for learning parallel-jaw grasping from synthetic stereo images", please refer to it if you are using the data.

Photorealistic indoor dataset designed to enable the application of deep learning techniques to a wide variety of robotic vision problems. The RobotriX consists of hyperrealistic indoor scenes which are explored by robot agents which also interact with objects in a visually realistic manner in that simulated world. Source: The RobotriX: An eXtremely Photorealistic and Very-Large-Scale Indoor Dataset of Sequences with Robot Trajectories and Interactions

The ViCoS Towel Dataset is a state-of-the-art benchmark for grasp point localization on cloth objects, specifically towels. Designed to advance research in robotic grasping and perception for textile objects, this dataset includes a collection of 8,000 high-resolution RGB-D images (1920×1080) captured with a Kinect V2 under a variety of conditions. Each image provides detailed depth information, making it ideal for training deep learning models and conducting thorough benchmarking. Object Diversity The dataset features 10 types of towels from the Household Cloth Objects. including various sizes and patterns such as big towels, checkered rags, and waffle rags. Each towel's corners are treated as potential grasp points for detection and localization. Object Positions Towels were positioned in 10 different configurations on a tabletop to simulate a range of visibility scenarios, from fully spread-out to crumpled and folded, providing diverse grasping conditions. Background Variability To enhance background variability, we used five different tabletop cloths or objects, including festive tablecloths and patterned tablecloths. This variation helps test algorithms under different background conditions. Lighting Conditions Images were captured under eight distinct lighting setups, ranging from no light to fully illuminated scenes with various shadow intensities. This variety helps in evaluating the robustness of grasping models under different lighting scenarios. Clutter Both cluttered and uncluttered scenes were included to create challenging conditions with occlusions. Various clutter items were added on the desk or directly on the towels to simulate realistic scenarios. ### Annotations Each visible corner of the towels in the images has been manually annotated as a potential grasping point. Annotations include a point label and the angle-of-approach, providing a total of 20,784 annotated points with corresponding angles. ### Training and Testing Splits The dataset is divided into training and testing subsets. Two towels (checkered rag small and cotton napkin) and one background (festive tablecloth) were reserved for testing, ensuring that these elements are not present in the training set. The dataset includes 5,120 training images and 2,880 testing images, covering all object configurations, lighting conditions, and clutter scenarios. ### Synthetic Training Data In addition to the real images, the dataset includes 12,000 synthetic images generated using the MuJoCo simulation environment. These synthetic images depict towels in varied positions and conditions, with different textures and lighting settings, providing further diversity for training purposes. For more details and access to the dataset, visit GitHub repository. CeDiRNet-3DoF: Center Direction Network for Grasping Point Localization on Cloths!

Estimating 6D object poses is a major challenge in 3D computer vision. Building on successful instance-level approaches, research is shifting towards category-level pose estimation for practical applications. Current categorylevel datasets, however, fall short in annotation quality and pose variety. Addressing this, we introduce HouseCat6D, a new category-level 6D pose dataset. It features 1) multimodality with Polarimetric RGB and Depth (RGBD+P), 2) encompasses 194 diverse objects across 10 household categories, including two photometrically challenging ones, and 3) provides high-quality pose annotations with an error range of only 1.35 mm to 1.74 mm. The dataset also includes 4) 41 large-scale scenes with comprehensive viewpoint and occlusion coverage, 5) a checkerboard-free environment, and 6) dense 6D parallel-jaw robotic grasp annotations. Additionally, we present benchmark results for leading category-level pose estimation networks.

The KIT Whole-Body Human Motion Database is a large-scale dataset of whole-body human motion with methods and tools, which allows a unifying representation of captured human motion, and efficient search in the database, as well as the transfer of subject-specific motions to robots with different embodiments. Captured subject-specific motion is normalized regarding the subject’s height and weight by using a reference kinematics and dynamics model of the human body, the master motor map (MMM). In contrast with previous approaches and human motion databases, the motion data in this database consider not only the motions of the human subject but the position and motion of objects with which the subject is interacting as well. In addition to the description of the MMM reference model, See the paper for procedures and techniques used for the systematic recording, labeling, and organization of human motion capture data, object motions as well as the subject–object relations. Source: https://motion-database.humanoids.kit.edu/ Image Source: https://motion-database.humanoids.kit.edu/

Dataset: RGB-D Images for Real-World and Synthetic Object Scenes This dataset consists of both real-world and synthetic RGB-D images, designed for object detection, classification, and segmentation tasks, particularly for primitive shape recognition. ## Real-World Data - Objects: 50 distinct objects captured using a Kinect camera. - Scenes: The dataset includes both single objects and piles of objects stacked over each other. - Images: Approximately 300 RGB-D images have been collected. - Data Format: RGB images paired with corresponding depth images. ## Synthetic Data - Simulator: The synthetic dataset was automatically generated using the CoppeliaSim simulator. - Objects: Only primitive shapes, including cuboid, semisphere, sphere, cylinder, stick, ring, and cone. - Scenes: Objects are dropped over each other to form complex scenes. - Images: The dataset contains 10,000 unique RGB-D images generated by the simulator. ## Labels Both the real-world and synthetic datasets are labeled with geometric primitive shape classes and boundaries, including: - Cuboid - Semisphere - Sphere - Cylinder - Stick - Ring - Cone This dataset is designed to support research in grasp detection, object recognition, and scene understanding using RGB-D data.