3D Hand Pose Estimation

FreiHAND is a 3D hand pose dataset which records different hand actions performed by 32 people. For each hand image, MANO-based 3D hand pose annotations are provided. It currently contains 32,560 unique training samples and 3960 unique samples for evaluation. The training samples are recorded with a green screen background allowing for background removal. In addition, it applies three different post processing strategies to training samples for data augmentation. However, these post processing strategies are not applied to evaluation samples. Source: Knowledge as Priors: Cross-Modal Knowledge Generalizationfor Datasets without Superior Knowledge Image Source: https://lmb.informatik.uni-freiburg.de/resources/datasets/FreihandDataset.en.html

DexYCB is a dataset for capturing hand grasping of objects. It can be used three relevant tasks: 2D object and keypoint detection, 6D object pose estimation, and 3D hand pose estimation. The dataset was built using 20 objects from the YCB-Video dataset, and consists of multiple trials from 10 subjects. For each trial, there is a target object with 2 to 4 other objects placed on a table. The subject is asked to start from a relaxed pose, pick up the target object, and hold it in the air. Some subjects were asked to pretend to hand over the object to someone across from them. Each action is recorded for 3 seconds, repeating the trial 5 times for each target object, each time with a random set of accompanied objects and placement. In total there are 100 trials per subject, and 1,000 trials in total for all subjects. Source: DexYCB: A Benchmark for Capturing Hand Grasping of Objects

GRAB is a dataset of full-body motions interacting and grasping 3D objects. It contains accurate finger and facial motions as well as the contact between the objects and body. It contains 5 male and 5 female participants and 4 different motion intents. The GRAB dataset also contains binary contact maps between the body and objects. Source: https://github.com/otaheri/GRAB Image Source: https://github.com/otaheri/GRAB

AGORA is a synthetic human dataset with high realism and accurate ground truth. It consists of around 14K training and 3K test images by rendering between 5 and 15 people per image using either image-based lighting or rendered 3D environments, taking care to make the images physically plausible and photoreal. In total, AGORA contains 173K individual person crops. AGORA provides (1) SMPL/SMPL-X parameters and (2) segmentation masks for each subject in images.

The InterHand2.6M dataset is a large-scale real-captured dataset with accurate GT 3D interacting hand poses, used for 3D hand pose estimation The dataset contains 2.6M labeled single and interacting hand frames. Source: InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image

A hand-object interaction dataset with 3D pose annotations of hand and object. The dataset contains 66,034 training images and 11,524 test images from a total of 68 sequences. The sequences are captured in multi-camera and single-camera setups and contain 10 different subjects manipulating 10 different objects from YCB dataset. The annotations are automatically obtained using an optimization algorithm. The hand pose annotations for the test set are withheld and the accuracy of the algorithms on the test set can be evaluated with standard metrics using the CodaLab challenge submission(see project page). The object pose annotations for the test and train set are provided along with the dataset.

Curates a dataset of SMPL-X fits on in-the-wild images. Source: Monocular Expressive Body Regression through Body-Driven Attention

ARCTIC is a dataset of free-form interactions of hands and articulated objects. ARCTIC has 1.2M images paired with accurate 3D meshes for both hands and for objects that move and deform over time. The dataset also provides hand-object contact information.

3D Hand Pose is a multi-view hand pose dataset consisting of color images of hands and different kind of annotations for each: the bounding box and the 2D and 3D location on the joints in the hand. Source: Large-scale Multiview 3D Hand Pose Dataset

First-Person Hand Action Benchmark is a collection of RGB-D video sequences comprised of more than 100K frames of 45 daily hand action categories, involving 26 different objects in several hand configurations. Source: First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations

We present a comprehensive framework for egocentric interaction recognition using markerless 3D annotations of two hands manipulating objects. To this end, we propose a method to create a unified dataset for egocentric 3D interaction recognition. Our method produces annotations of the 3D pose of two hands and the 6D pose of the manipulated objects, along with their interaction labels for each frame. Our dataset, called H2O (2 Hands and Objects), provides synchronized multi-view RGB-D images, interaction labels, object classes, ground-truth 3D poses for left & right hands, 6D object poses, ground-truth camera poses, object meshes and scene point clouds. To the best of our knowledge, this is the first benchmark that enables the study of first-person actions with the use of the pose of both left and right hands manipulating objects and presents an unprecedented level of detail for egocentric 3D interaction recognition. We further propose the method to predict interaction classes by estimating the 3D pose of two hands and the 6D pose of the manipulated objects, jointly from RGB images. Our method models both inter- and intra-dependencies between both hands and objects by learning the topology of a graph convolutional network that predicts interactions. We show that our method facilitated by this dataset establishes a strong baseline for joint hand-object pose estimation and achieves state-of-the-art accuracy for first person interaction recognition.

The EgoDexter dataset provides both 2D and 3D pose annotations for 4 testing video sequences with 3190 frames. The videos are recorded with body-mounted camera from egocentric viewpoints and contain cluttered backgrounds, fast camera motion, and complex interactions with various objects. Fingertip positions were manually annotated for 1485 out of 3190 frames. Source: Hand Pose Estimation via Latent 2.5D Heatmap Regression Image Source: https://handtracker.mpi-inf.mpg.de/projects/OccludedHands/EgoDexter.htm

The HO-3D v3 is the version 3 of the HO-3D dataset with more accurate hand-object poses. HO-3D v3 provides more accurate annotations for both the hand and object poses thus resulting in better estimates of contact regions between the hand and the object. The table below shows the statistics of the HO-3D v2 compared to the HO-3D v3 datasets. | Dataset | Train Set | Test Set | #Images | #Objects | #Subjects | Release Date | |-----------|-----------|----------|---------|----------|-----------|--------------| | HO-3D v2 | 66,034 | 11,524 | 10 | 10 | 9 | 9 Jan 2020 | | HO-3D v3 | 83,325 | 20,137 | 10 | 10 | 1 | 1 Jul 2021 | Source: Official Dataset Image Source: Official Dataset

Human3.6M 3D WholeBody (H3WB) is a large scale dataset with 133 whole-body keypoint annotations on 100K images, made possible by a new multi-view pipeline. It is designed for the three new tasks : i) 3D whole-body pose lifting from 2D complete whole-body pose, ii) 3D whole-body pose lifting from 2D incomplete whole-body pose, iii) 3D whole-body pose estimation from a single RGB image. Source: H3WB: Human3.6M 3D WholeBody Dataset and Benchmark Image Source: https://arxiv.org/pdf/2211.15692v1.pdf

The HInt dataset is frequently used as a <b>generalizability benchmark</b> for 3D Hand Reconstruction. It features three data subsets: HInt-NewDays, HInt-VISOR and HInt-Ego4D subsets and it aims to complement existing datasets used for training and evaluation 3D hand pose estimation. HInt annotates 2D keypoint locations and occlusion labels for 21 keypoints on the hand. It is built off of 3 existing datasets (Hands23, Epic-Kitchens VISOR, and Ego4D) and provides annotations for images from the three existing datasets. Source: Reconstructing Hands in 3D with Transformers Image Source: Official Dataset

The SynthHands dataset is a dataset for hand pose estimation which consists of real captured hand motion retargeted to a virtual hand with natural backgrounds and interactions with different objects. The dataset contains data for male and female hands, both with and without interaction with objects. While the hand and foreground object are synthtically generated using Unity, the motion was obtained from real performances as described in the accompanying paper. In addition, real object textures and background images (depth and color) were used. Ground truth 3D positions are provided for 21 keypoints of the hand. Source: Egocentric 6-DoF Tracking of Small Handheld Objects Image Source: https://handtracker.mpi-inf.mpg.de/projects/OccludedHands/SynthHands.htm

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

Robot grasping is often formulated as a learning problem. With the increasing speed and quality of physics simulations, generating large-scale grasping data sets that feed learning algorithms is becoming more and more popular. An often overlooked question is how to generate the grasps that make up these data sets. In this paper, we review, classify, and compare different grasp sampling strategies. Our evaluation is based on a fine-grained discretization of SE(3) and uses physics-based simulation to evaluate the quality and robustness of the corresponding parallel-jaw grasps. Specifically, we consider more than 1 billion grasps for each of the 21 objects from the YCB data set. This dense data set lets us evaluate existing sampling schemes w.r.t. their bias and efficiency. Our experiments show that some popular sampling schemes contain significant bias and do not cover all possible ways an object can be grasped.

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

The ICVL dataset is a hand pose estimation dataset that consists of 330K training frames and 2 testing sequences with each 800 frames. The dataset is collected from 10 different subjects with 16 hand joint annotations for each frame. Source: AWR: Adaptive Weighting Regression for 3D Hand Pose Estimation Image Source: Tang et al.; Latent Regression Forest: Structured Estimation of 3D Hand Poses

https://syncandshare.lrz.de/getlink/fi9EZb33KiSAJ5rLHAkhg7/ffhnet-data.zip

MuViHand is a dataset for 3D Hand Pose Estimation that consists of multi-view videos of the hand along with ground-truth 3D pose labels. The dataset includes more than 402,000 synthetic hand images available in 4,560 videos. The videos have been simultaneously captured from six different angles with complex backgrounds and random levels of dynamic lighting. The data has been captured from 10 distinct animated subjects using 12 cameras in a semi-circle topology.

ThermoHands is the first benchmark dataset specifically designed for egocentric 3D hand pose estimation from thermal images. It addresses the challenges of hand pose estimation in low-light conditions and when the hand is occluded by gloves or other wearables—scenarios where traditional RGB or NIR-based systems struggle. The dataset contains approximately 96,000 synchronized multi-view, multi-spectral images collected from 28 participants under five varied scenarios involving changes in lighting, handwear, and background environments. Each sample includes thermal, NIR, RGB, and depth images, captured using a custom-built head-mounted sensor platform (HMSP) and a multi-view exocentric setup. Ground-truth 3D hand poses and shapes are automatically annotated using multi-view optimization based on the MANO model. Key features: Multi-spectral data: Egocentric and exocentric views of thermal, NIR, RGB, and depth modalities Rich annotations: 3D hand pose and shape ground truth via an automated optimization pipeline Realistic interactions: Both hand-object and hand-virtual interactions in diverse everyday scenarios Motivation and Use Cases: ThermoHands enables research into robust, privacy-preserving hand pose estimation for XR, AR/VR, and human-computer interaction applications. It is particularly useful for scenarios involving poor illumination, occlusion by gloves, or the need for passive sensing (e.g., avoiding NIR interference or power constraints). This dataset also introduces TherFormer, a novel transformer-based baseline tailored to thermal input, and benchmarks multiple SOTA methods under cross-spectral settings. Paper: https://arxiv.org/pdf/2403.09871 Project page: https://thermohands.github.io/ Code and models: https://github.com/LawrenceZ22/ThermoHands

A large-scale hand pose dataset, collected using a novel capture method. Source: BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis

Includes several sets of synthetic stereo images labelled with grasp rectangles representing parallel-jaw grasps (Cornell-like format). The set was introduced in the range of the ICAR paper "Automatic generation of realistic training data for learning parallel-jaw grasping from synthetic stereo images", please refer to it if you are using the data.

3D hand pose data set created using stereo camera - contains 18,000 RGB images and paired depth images - 3D positions of hand joints (21 joints)

The dataset consists of images of Human palms captured using a mobile phone. The images have been taken in a real-world scenario like holding objects or performing simple gestures. The dataset has a wide variety of variations like illumination, distances etc. It consists of images of 3 main gestures: Frontal-open palm, Back open palm and fist with the wrist. It also has a lot of images with people wearing gloves. Dataset Features: - Captured by 4000+ unique users - Covers wide variety of palm images in indoor and outdoor scenes - Images of palm with gloves - Captured by male and female - Distributed age groups like teenage, adults and old - Captured using mobile phones - Highly diverse - Various lighting conditions like day, night, indoor and outdoor - Outdoor scene with a variety of viewpoints Annotations Format: - Classification and detection annotations available - Multiple category annotations possible - COCO, PASCAL VOC and YOLO formats To download full datasets or to submit a request for your dataset needs, please ping us at (sales@datacluster.ai)[sales@datacluster.ai] Visit www.datacluster.ai to know more.