Pose Estimation

Dataset produced for the SAPIEN simulation environment. From the website: "PartNet-Mobility dataset is a collection of 2K articulated objects with motion annotations and rendernig material. The dataset powers research for generalizable computer vision and manipulation. The dataset is a continuation of ShapeNet and PartNet. "

Rendered Hand Pose (RHD) is a dataset for hand pose estimation. It provides segmentation maps with 33 classes: three for each finger, palm, person, and background. The 3D kinematic model of the hand provides 21 keypoints per hand: 4 keypoints per finger and one keypoint close to the wrist.

GRAB is a dataset of full-body motions interacting and grasping 3D objects. It contains accurate finger and facial motions as well as the contact between the objects and body. It contains 5 male and 5 female participants and 4 different motion intents. The GRAB dataset also contains binary contact maps between the body and objects. Source: https://github.com/otaheri/GRAB Image Source: https://github.com/otaheri/GRAB

MuPoTs-3D (Multi-person Pose estimation Test Set in 3D) is a dataset for pose estimation composed of more than 8,000 frames from 20 real-world scenes with up to three subjects. The poses are annotated with a 14-point skeleton model. Source: DOPE: Distillation Of Part Experts for whole-body 3D pose estimation in the wild Image Source: http://gvv.mpi-inf.mpg.de/projects/SingleShotMultiPerson/

AGORA is a synthetic human dataset with high realism and accurate ground truth. It consists of around 14K training and 3K test images by rendering between 5 and 15 people per image using either image-based lighting or rendered 3D environments, taking care to make the images physically plausible and photoreal. In total, AGORA contains 173K individual person crops. AGORA provides (1) SMPL/SMPL-X parameters and (2) segmentation masks for each subject in images.

InLoc is a dataset with reference 6DoF poses for large-scale indoor localization. Query photographs are captured by mobile phones at a different time than the reference 3D map, thus presenting a realistic indoor localization scenario. Source: InLoc: Indoor Visual Localization with Dense Matching and View Synthesis

REAL275 is a benchmark for category-level pose estimation. It contains 4300 training frames, 950 validation and 2750 for testing across 18 different real scenes.

This dataset focuses on heavily occluded human with comprehensive annotations including bounding-box, humans pose and instance mask. This dataset contains 13,360 elaborately annotated human instances within 5081 images. With average 0.573 MaxIoU of each person, OCHuman is the most complex and challenging dataset related to human. Source: https://github.com/liruilong940607/OCHumanApi Image Source: https://github.com/liruilong940607/OCHumanApi

VOCASET is a 4D face dataset with about 29 minutes of 4D scans captured at 60 fps and synchronized audio. The dataset has 12 subjects and 480 sequences of about 3-4 seconds each with sentences chosen from an array of standard protocols that maximize phonetic diversity. Source: timzhang642

The TotalCapture dataset consists of 5 subjects performing several activities such as walking, acting, a range of motion sequence (ROM) and freestyle motions, which are recorded using 8 calibrated, static HD RGB cameras and 13 IMUs attached to head, sternum, waist, upper arms, lower arms, upper legs, lower legs and feet, however the IMU data is not required for our experiments. The dataset has publicly released foreground mattes and RGB images. Ground-truth poses are obtained using a marker-based motion capture system, with the markers are <5mm in size. All data is synchronised and operates at a framerate of 60Hz, providing ground truth poses as joint positions. Source: Semantic Estimation of 3D Body Shape and Pose using Minimal Cameras Image Source: https://cvssp.org/data/totalcapture/

Inferring human-scene contact (HSC) is the first step toward understanding how humans interact with their surroundings. While detecting 2D human-object interaction (HOI) and reconstructing 3D human pose and shape (HPS) have enjoyed significant progress, reasoning about 3D human-scene contact from a single image is still challenging. Existing HSC detection methods consider only a few types of predefined contact, often reduce body and scene to a small number of primitives, and even overlook image evidence. To predict human-scene contact from a single image, we address the limitations above from both data and algorithmic perspectives. We capture a new dataset called RICH for “Real scenes, Interaction, Contact and Humans.” RICH contains multiview outdoor/indoor video sequences at 4K resolution, ground-truth 3D human bodies captured using markerless motion capture, 3D body scans, and high resolution 3D scene scans. A key feature of RICH is that it also contains accurate vertex-level contact labels on the body

The InterHand2.6M dataset is a large-scale real-captured dataset with accurate GT 3D interacting hand poses, used for 3D hand pose estimation The dataset contains 2.6M labeled single and interacting hand frames. Source: InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image

UAV-Human is a large dataset for human behavior understanding with UAVs. It contains 67,428 multi-modal video sequences and 119 subjects for action recognition, 22,476 frames for pose estimation, 41,290 frames and 1,144 identities for person re-identification, and 22,263 frames for attribute recognition. The dataset was collected by a flying UAV in multiple urban and rural districts in both daytime and nighttime over three months, hence covering extensive diversities w.r.t subjects, backgrounds, illuminations, weathers, occlusions, camera motions, and UAV flying attitudes. This dataset can be used for UAV-based human behavior understanding, including action recognition, pose estimation, re-identification, and attribute recognition.

The FLIC dataset contains 5003 images from popular Hollywood movies. The images were obtained by running a state-of-the-art person detector on every tenth frame of 30 movies. People detected with high confidence (roughly 20K candidates) were then sent to the crowdsourcing marketplace Amazon Mechanical Turk to obtain ground truth labelling. Each image was annotated by five Turkers to label 10 upper body joints. The median-of-five labelling was taken in each image to be robust to outlier annotation. Finally, images were rejected manually by if the person was occluded or severely non-frontal. Source: https://bensapp.github.io/flic-dataset.html Image Source: https://www.tensorflow.org/datasets/catalog/flic

BEDLAM is a large-scale synthetic video dataset designed to train and test algorithms on the task of 3D human pose and shape estimation (HPS). It contains diverse body shapes, skin tones, and motions. The clothing is realistically simulated on the moving bodies using commercial clothing physics simulation.

The dataset contains over 15K images of 20 people (6 females and 14 males - 4 people were recorded twice). For each frame, a depth image, the corresponding rgb image (both 640x480 pixels), and the annotation is provided. The head pose range covers about +-75 degrees yaw and +-60 degrees pitch. Ground truth is provided in the form of the 3D location of the head and its rotation. Source: https://www.kaggle.com/kmader/biwi-kinect-head-pose-database Image Source: https://icu.ee.ethz.ch/research/datsets.html

Accurate modeling of priors over 3D human pose is fundamental to many problems in computer vision. Most previous priors are either not general enough for the diverse nature of human poses or not restrictive enough to avoid invalid 3D poses. We propose a physically-motivated prior that only allows anthropometrically valid poses and restricts the ones that are invalid. One can use joint-angle limits to evaluate whether two connected bones are valid or not. However, it is established in biomechanics that there are dependencies in joint-angle limits between certain pair of bones. For example how much one can flex one’s arm depends on whether it is in front of, or behind, the back. Medical textbooks only provide joint-angle limits in a few positions and the complete configuration of pose-dependent joint-angle limits for the full body is unknown.

AP-10K is the first large-scale benchmark for general animal pose estimation, to facilitate the research in animal pose estimation. AP-10K consists of 10,015 images collected and filtered from 23 animal families and 60 species following the taxonomic rank and high-quality keypoint annotations labeled and checked manually. Source: AP-10K: A Benchmark for Animal Pose Estimation in the Wild Image source: https://arxiv.org/pdf/2108.12617v1.pdf

The Campus and Shelf datasets were presented in the paper 3D Pictorial Structures for Multiple Human Pose Estimation. The first dataset shows persons walking and talking in front of a building, and the second up to four persons assembling a shelf.

GazeFollow is a large-scale dataset annotated with the location of where people in images are looking. It uses several major datasets that contain people as a source of images: 1, 548 images from SUN, 33, 790 images from MS COCO, 9, 135 images from Actions 40, 7, 791 images from PASCAL, 508 images from the ImageNet detection challenge and 198, 097 images from the Places dataset. This concatenation results in a challenging and large image collection of people performing diverse activities in many everyday scenarios. Source: GazeFollow

A hand-object interaction dataset with 3D pose annotations of hand and object. The dataset contains 66,034 training images and 11,524 test images from a total of 68 sequences. The sequences are captured in multi-camera and single-camera setups and contain 10 different subjects manipulating 10 different objects from YCB dataset. The annotations are automatically obtained using an optimization algorithm. The hand pose annotations for the test set are withheld and the accuracy of the algorithms on the test set can be evaluated with standard metrics using the CodaLab challenge submission(see project page). The object pose annotations for the test and train set are provided along with the dataset.

JTA is a dataset for people tracking in urban scenarios by exploiting a photorealistic videogame. It is up to now the vastest dataset (about 500.000 frames, almost 10 million body poses) of human body parts for people tracking in urban scenarios. Source: Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World

EgoBody dataset is a novel large-scale dataset for egocentric 3D human pose, shape and motions under interactions in complex 3D scenes.

MuCo-3DHP is a large scale training data set showing real images of sophisticated multi-person interactions and occlusions. Source: Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB

The LM (Linemod) dataset is a valuable resource introduced by Stefan Hinterstoisser and colleagues in their research on model-based training, detection, and pose estimation of texture-less 3D objects in heavily cluttered scenes¹. Let's delve into the details: 1. Purpose and Context: - The primary goal of the LM dataset is to facilitate the development and evaluation of methods for detecting and estimating the 6 degrees-of-freedom pose of texture-less 3D objects. - It specifically targets scenarios where objects lack distinctive textures and are embedded in complex backgrounds with occlusions. 2. Methodology: - The dataset builds upon the LINEMOD approach, which combines depth and color information to create templates representing different views of an object. - LINEMOD templates are learned from 3D models and serve as a basis for object detection. - The initial LINEMOD method had limitations, including online template learning and approximate pose estimation. 3. Improvements and Contributions: - Hinterstoisser et al. enhance LINEMOD by incorporating accurate 3D models of objects. - Their approach leverages the 3D model to address the shortcomings of the original LINEMOD. - Notable improvements include better pose estimation and reduced false positives. - The proposed framework is suitable for robotics applications, such as object manipulation. 4. Dataset Details: - The LM dataset consists of 15 registered video sequences, each containing over 1100 frames. - These sequences feature 15 different texture-less household objects. - Objects in the dataset exhibit discriminative color, shape, and size characteristics. - Researchers can use this dataset to evaluate and compare their methods for object detection and pose estimation. LM Dataset Example In summary, the LM dataset provides a valuable benchmark for advancing the field of 6D object pose estimation, especially in scenarios where texture information is limited¹². Researchers can access this dataset to test and refine their algorithms, ultimately contributing to advancements in robotics and machine vision. (1) Model Based Training, Detection and Pose ... - Stefan HINTERSTOISSER. http://stefan-hinterstoisser.com/papers/hinterstoisser2012accv.pdf. (2) Datasets - BOP: Benchmark for 6D Object Pose Estimation. https://bop.felk.cvut.cz/datasets/. (3) paroj/linemod_dataset: Hinterstoisser et al. ACCV12 dataset - GitHub. https://github.com/paroj/linemod_dataset.

COCO-WholeBody is an extension of COCO dataset with whole-body annotations. There are 4 types of bounding boxes (person box, face box, left-hand box, and right-hand box) and 133 keypoints (17 for body, 6 for feet, 68 for face and 42 for hands) annotations for each person in the image. Source: Whole-Body Human Pose Estimation in the Wild Image source: https://arxiv.org/pdf/2007.11858v1.pdf

EMDB contains in-the-wild videos of human activity recorded with a hand-held iPhone. It features reference SMPL body pose and shape parameters, as well as global body root and camera trajectories. The reference 3D poses were obtained by jointly fitting SMPL to 12 body-worn electromagnetic sensors and image data. For the latter we fit a neural implicit avatar model to allow for a dense pixel-wise fitting objective. EMDB contains: * 81 sequences * 105 000 frames * 10 actors (5 female, 5 male) * Global camera trajectories * SMPL pose and shape parameters * 2D Keypoints The dataset can be used to evaluate the following tasks: * Camera-relative 3D human pose and shape estimation from monocular videos. * Global 3D human pose and shape estimation including camera trajectories from monocular videos. * Human motion prediction.

A novel benchmark and dataset for the evaluation of image-based garment reconstruction systems. Deep Fashion3D contains 2078 models reconstructed from real garments, which covers 10 different categories and 563 garment instances. It provides rich annotations including 3D feature lines, 3D body pose and the corresponded multi-view real images. In addition, each garment is randomly posed to enhance the variety of real clothing deformations. Source: Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images

Curates a dataset of SMPL-X fits on in-the-wild images. Source: Monocular Expressive Body Regression through Body-Driven Attention

The Gaming 3D Dataset (G3D) focuses on real-time action recognition in a gaming scenario. It contains 10 subjects performing 20 gaming actions: “punch right”, “punch left”, “kick right”, “kick left”, “defend”, “golf swing”, “tennis swing forehand”, “tennis swing backhand”, “tennis serve”, “throw bowling ball”, “aim and fire gun”, “walk”, “run”, “jump”, “climb”, “crouch”, “steer a car”, “wave”, “flap” and “clap”. Source: Skeleton Based Action Recognition Using Translation-Scale Invariant Image Mapping And Multi-Scale Deep CNN Image Source: G3D: A gaming action dataset and real time action recognition evaluation framework

The MannequinChallenge Dataset (MQC) provides in-the-wild videos of people in static poses while a hand-held camera pans around the scene. The dataset consists of three splits for training, validation and testing. Source: Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild Image Source: https://google.github.io/mannequinchallenge/www/index.html

The Vid4 dataset is generally used for testing video super-resolution. It consists of four sequences: walk (740x480, 47 frames), foliage (740x480, 49 frames), city (704x576, 34 frames), and calendar (720x576, 41 frames).

Animal Kingdom is a large and diverse dataset that provides multiple annotated tasks to enable a more thorough understanding of natural animal behaviors. The wild animal footage used in the dataset records different times of the day in an extensive range of environments containing variations in backgrounds, viewpoints, illumination and weather conditions. More specifically, the dataset contains 50 hours of annotated videos to localize relevant animal behavior segments in long videos for the video grounding task, 30K video sequences for the fine-grained multi-label action recognition task, and 33K frames for the pose estimation task, which correspond to a diverse range of animals with 850 species across 6 major animal classes.

HomebrewedDB is a dataset for 6D pose estimation mainly targeting training from 3D models (both textured and textureless), scalability, occlusions, and changes in light conditions and object appearance. The dataset features 33 objects (17 toy, 8 household and 8 industry-relevant objects) over 13 scenes of various difficulty. It also consists of a set of benchmarks to test various desired detector properties, particularly focusing on scalability with respect to the number of objects and resistance to changing light conditions, occlusions and clutter.

Our dataset was made of videos from MSU Video Upscalers Benchmark Dataset, MSU Video Super-Resolution Benchmark Dataset and MSU Super-Resolution for Video Compression Benchmark Dataset. Dataset consists of real videos (were filmed with 2 cameras), video games footages, movies, cartoons, dynamic ads. How we brought our dataset closer to completeness? * The dataset covers a large number of use cases in the field of SR due to the large number of content types * The dataset contains videos with completely different resolutions, FPS values: 8, 24, 25, 30, 60, as well as high and low spatio-temporal complexity * Distorted videos were obtained using 46 SR methods, some of them were preprocessed with 5 codecs: aomenc, vvenc, x264, x265, uavs3es with different bitrates and qp values * The dataset was manually checked for redundancy Videos from benchmarks are FullHD video crops, since the subjective comparison was made on crops. Therefore, the resolution of all videos in the received dataset is low.

A three million frame, multi-view, furniture assembly video dataset that includes depth, atomic actions, object segmentation, and human pose. Source: The IKEA ASM Dataset: Understanding People Assembling Furniture through Actions, Objects and Pose