Pose Estimation

This is a pose estimation dataset, consisting of symmetric 3D shapes where multiple orientations are visually indistinguishable. The challenge is to predict all equivalent orientations when only one orientation is paired with each image during training (as is the scenario for most pose estimation datasets). In contrast to most pose estimation datasets, the full set of equivalent orientations is available for evaluation. There are eight shapes total, each rendered from 50,000 viewpoints distributed uniformly at random over the full space of 3D rotations. Five of the shapes are featureless -- tetrahedron, cube, icosahedron, cone, and cylinder. Of those, the three Platonic solids (tetrahedron, cube, icosahedron) are annotated with their 12, 24, and 60 discrete symmetries, respectively. The cone and cylinder are annotated with their continuous symmetries discretized at 1 degree intervals. These symmetries are provided for evaluation; the intended supervision is only a single rotation with each image. The remaining three shapes are marked with a distinguishing feature. There is a tetrahedron with one red-colored face, a cylinder with an off-center dot, and a sphere with an X capped by a dot. Whether or not the distinguishing feature is visible, the space of possible orientations is reduced. We do not provide the set of equivalent rotations for these shapes. Each example contains of - the 224x224 RGB image - a shape index so that the dataset may be filtered by shape. The indices correspond to: 0 = tetrahedron 1 = cube 2 = icosahedron 3 = cone 4 = cylinder 5 = marked tetrahedron 6 = marked cylinder 7 = marked sphere - the rotation used in the rendering process, represented as a 3x3 rotation matrix - the set of known equivalent rotations under symmetry, for evaluation. In the case of the three marked shapes, this is only the rendering rotation.

TAMPAR is a real-world dataset of parcel photos for tampering detection with annotations in COCO format. For details see the paper and for visual samples the project page. Features are: - >900 annotated real-world images with >2,700 visible parcel side surfaces - 6 different tampering types - 6 different distortion strengths

Data and experiments for motion-based extrinsic calibration using [trajectory_calibration]. The data was generated for evaluating hand-eye calibration algorithms in extrinsic sensor-to-sensor calibration. The repository is organised as follows: ├── calib │ └── *.csv # The output calibration values ├── kitti_tests │ └── ... # The trajectories generated from KITTI data ├── matlab │ └── ... # Matlab files to generate simulation data ├── simulation_tests │ └── ... # The generated simulation data ├── *.py # Scripts to run the calibration experiments └── README.md If you use the data in an academic context, please cite: @article{valimaki2023, author = {Välimäki, Tuomas and Garigipati, Bharath and Ghabcheloo, Reza}, title = {Motion-Based Extrinsic Sensor-to-Sensor Calibration: Effect of Reference Frame Selection for New and Existing Methods}, journal = {Sensors}, volume = {23}, year = {2023}, number = {7}, pages = {3740}, url = {https://www.mdpi.com/1424-8220/23/7/3740}, issn = {1424-8220}, doi = {10.3390/s23073740} } [trajectory_calibration]: https://github.com/tau-alma/trajectory_calibration

The ViCoS Towel Dataset is a state-of-the-art benchmark for grasp point localization on cloth objects, specifically towels. Designed to advance research in robotic grasping and perception for textile objects, this dataset includes a collection of 8,000 high-resolution RGB-D images (1920×1080) captured with a Kinect V2 under a variety of conditions. Each image provides detailed depth information, making it ideal for training deep learning models and conducting thorough benchmarking. Object Diversity The dataset features 10 types of towels from the Household Cloth Objects. including various sizes and patterns such as big towels, checkered rags, and waffle rags. Each towel's corners are treated as potential grasp points for detection and localization. Object Positions Towels were positioned in 10 different configurations on a tabletop to simulate a range of visibility scenarios, from fully spread-out to crumpled and folded, providing diverse grasping conditions. Background Variability To enhance background variability, we used five different tabletop cloths or objects, including festive tablecloths and patterned tablecloths. This variation helps test algorithms under different background conditions. Lighting Conditions Images were captured under eight distinct lighting setups, ranging from no light to fully illuminated scenes with various shadow intensities. This variety helps in evaluating the robustness of grasping models under different lighting scenarios. Clutter Both cluttered and uncluttered scenes were included to create challenging conditions with occlusions. Various clutter items were added on the desk or directly on the towels to simulate realistic scenarios. ### Annotations Each visible corner of the towels in the images has been manually annotated as a potential grasping point. Annotations include a point label and the angle-of-approach, providing a total of 20,784 annotated points with corresponding angles. ### Training and Testing Splits The dataset is divided into training and testing subsets. Two towels (checkered rag small and cotton napkin) and one background (festive tablecloth) were reserved for testing, ensuring that these elements are not present in the training set. The dataset includes 5,120 training images and 2,880 testing images, covering all object configurations, lighting conditions, and clutter scenarios. ### Synthetic Training Data In addition to the real images, the dataset includes 12,000 synthetic images generated using the MuJoCo simulation environment. These synthetic images depict towels in varied positions and conditions, with different textures and lighting settings, providing further diversity for training purposes. For more details and access to the dataset, visit GitHub repository. CeDiRNet-3DoF: Center Direction Network for Grasping Point Localization on Cloths!

To evaluate the performance on 4K burst images/video, we collect several clips from website. The dataset can be download from : https://drive.google.com/file/d/1YDljUONvyKUO24smTx__CUH_4Zxhle09/view?usp=sharing

Vinegar Fly is a pose estimation dataset for fruit flies.

Data used for the paper Combining Motion Matching and Orientation Prediction to Animate Avatars for Consumer-Grade VR Devices. MMData.zip contains the necessary files to execute the project in Unity. Use only following the instructions on the GitHub project. MMVR_Dataset.zip contains all .bvh files used for training the orientation prediction network. All files are captured with an Xsens Awinda motion capture system while using Virtual Reality. Visit GitHub for more information.

The YCB-Ev dataset contains synchronized RGB-D frames and event data that enables evaluating 6DoF object pose estimation algorithms using these modalities. This dataset provides ground truth 6DoF object poses for the same 21 YCB objects that were used in the YCB-Video (YCB-V) dataset, allowing for cross-dataset algorithm performance evaluation. The dataset consists of 21 synchronized event and RGB-D sequences, totalling 13,851 frames (7 minutes and 43 seconds of event data). Notably, 12 of these sequences feature the same object arrangement as the YCB-V subset used in the BOP challenge. Ground truth poses are generated by detecting objects in the RGB-D frames, interpolating the poses to align with the event timestamps, and then transferring them to the event coordinate frame using extrinsic calibration.

This dataset consists of both fractured and non-fractured X-ray images encompassing various anatomical regions of the body, such as the lower limb, upper limb, lumbar region, hips, knees, and more. It is organized into three main folders: train, test, and validation, each containing both fractured and non-fractured radiographic images. You can freely access the dataset via the following link: https://www.kaggle.com/datasets/bmadushanirodrigo/fracture-multi-region-x-ray-data/data

Grévy’s Zebra is an animal pose estimation dataset for zebras.

Estimating 6D object poses is a major challenge in 3D computer vision. Building on successful instance-level approaches, research is shifting towards category-level pose estimation for practical applications. Current categorylevel datasets, however, fall short in annotation quality and pose variety. Addressing this, we introduce HouseCat6D, a new category-level 6D pose dataset. It features 1) multimodality with Polarimetric RGB and Depth (RGBD+P), 2) encompasses 194 diverse objects across 10 household categories, including two photometrically challenging ones, and 3) provides high-quality pose annotations with an error range of only 1.35 mm to 1.74 mm. The dataset also includes 4) 41 large-scale scenes with comprehensive viewpoint and occlusion coverage, 5) a checkerboard-free environment, and 6) dense 6D parallel-jaw robotic grasp annotations. Additionally, we present benchmark results for leading category-level pose estimation networks.

The dataset consists of images of Human palms captured using a mobile phone. The images have been taken in a real-world scenario like holding objects or performing simple gestures. The dataset has a wide variety of variations like illumination, distances etc. It consists of images of 3 main gestures: Frontal-open palm, Back open palm and fist with the wrist. It also has a lot of images with people wearing gloves. Dataset Features: - Captured by 4000+ unique users - Covers wide variety of palm images in indoor and outdoor scenes - Images of palm with gloves - Captured by male and female - Distributed age groups like teenage, adults and old - Captured using mobile phones - Highly diverse - Various lighting conditions like day, night, indoor and outdoor - Outdoor scene with a variety of viewpoints Annotations Format: - Classification and detection annotations available - Multiple category annotations possible - COCO, PASCAL VOC and YOLO formats To download full datasets or to submit a request for your dataset needs, please ping us at (sales@datacluster.ai)[sales@datacluster.ai] Visit www.datacluster.ai to know more.

ICT-3DHP is collected using the Microsoft Kinect sensor and contains RGB images and depth maps of about 14k frames, divided in 10 sequences. The image resolution is 640 × 480 pixels. An hardware sensor (Polhemus Fastrack) is exploited to generate the ground truth annotation. The device is placed on a white cap worn by each subject, visible in both RGB and depth frames. Source: POSEidon: Face-from-Depth for Driver Pose Estimation

InfiniteRep is a synthetic, open-source dataset for fitness and physical therapy (PT) applications. It includes 1k videos of diverse avatars performing multiple repetitions of common exercises. It includes significant variation in the environment, lighting conditions, avatar demographics, and movement trajectories. From cadence to kinematic trajectory, each rep is done slightly differently -- just like real humans. InfiniteRep videos are accompanied by a rich set of pixel-perfect labels and annotations, including frame-specific repetition counts. The dataset features: + 100 videos per exercise, spanning 5 to 10 repetitions each (1,000 videos total) + 7 unique indoor scenes + Realistic environmental occlusion (+ corresponding labels) + Diverse lighting conditions + Varied body shape, skin tones, and clothing + Rich annotations for 2D and 3D supervision ## Exercises The dataset currently includes the following exercises: + Pushups + Alternating Bicep Curls (with dumbbells) + Delt Flys (with dumbbells) + Squats + Bird Dogs + Supermans + Bicycle Crunches + Leg Raises + Front Raises (with dumbbells) + Overhead Press (with dumbbells) ## Annotations The dataset includes the following annotations: + Bounding boxes + Segmentation masks + Keypoints + Joint angles (quaternions) + Percent occlusion + Avatar characteristics + Camera position + and more Want depth labels? They are not included in the dataset but we can send them to you. Email us at info@toinfinity.ai. ## Download Download the dataset: toinfinity.ai/infiniterep Github repo with additional documentation: https://github.com/toinfinityai/InfiniteRep ## Need more data? Infinity AI specializes in generating custom synthetic data. If you need more (or different data), drop us a line at info@toinfinity.ai (we read every email).

The LAAS Parkour dataset contains 28 RGB videos capturing human subjects performing four typical parkour techniques: safety-vault, kong vault, pull-up and muscle-up. These are highly dynamic motions with rich contact interactions with the environment. The dataset is provided with the ground truth 3D positions of 16 pre-defined human joints, together with the contact forces at the human subjects' hand and foot joints exerted by the environment.

Overview The goal: using simulation data to train neural networks to estimate the pose of a rover's camera with respect to a known target object The mission context: A simulated lunar surface, with lunar landers and lunar rovers. To accomplish their ressource extraction mission, the rovers must dig, transport and deliver regolith to a processing plant. For each of these tasks, a central need is for rovers to accurately estimate the relative pose both between themselves and with the landers. <img src="https://github.com/TeamL3/learned-pose-estimation/raw/main/misc/images/overview_low.png" alt="lunar surface overview" style="width:720px;"/> The dataset: - tf.data dataset ready for training (RGBD images and ground truth pose labels) - five different scenarios for relative pose estimation, some easier, some harder! <img src="https://github.com/TeamL3/learned-pose-estimation/raw/main/misc/images/dataset.png" alt="dataset samples" style="width:720px;"/> The code: Utilities for manipulating the dataset and calculating training metrics + example jupyter notebooks for data exploration and model training + more details on the dataset are available on github

The University of Padova Body Pose Estimation dataset (UNIPD-BPE) is an extensive dataset for multi-sensor body pose estimation containing both single-person and multi-person sequences with up to 4 interacting people A network with 5 Microsoft Azure Kinect RGB-D cameras is exploited to record synchronized high-definition RGB and depth data of the scene from multiple viewpoints, as well as to estimate the subjects’ poses using the Azure Kinect Body Tracking SDK. Simultaneously, full-body Xsens MVN Awinda inertial suits allow obtaining accurate poses and anatomical joint angles, while also providing raw data from the 17 IMUs required by each suit. All the cameras and inertial suits are hardware synchronized, while the relative poses of each camera with respect to the inertial reference frame are calibrated before each sequence to ensure maximum overlap of the two sensing systems outputs. The setup used allowed to record synchronized 3D poses of the persons on the scene both via Xsens’ inverse kinematics algorithm (inertial motion capture) and by exploiting the Azure Kinect Body tracking SDK (markerless motion capture), simultaneously. The additional raw data (RGB, depth, camera network configuration) allow the user to assess the performance of any custom markerless motion capture algorithm (based on RGB, depth, or both). Further analyses can be progressed by varying the number of cameras being used and/or their resolution and frame rate. Moreover, raw angular velocities, linear accelerations, magnetic fields, and orientations from each IMU allow to develop and test multimodal BPE approaches focused on merging visual and inertial data. Finally, the precise body dimensions of each subject are provided. They include body height, weight, and segment lengths measured before the beginning of a recording session. They were used to scale the Xsens biomechanical model, and also constitute a ground truth for assessing the markerless BPE accuracy on estimating each subject’s body dimensions. The recorded sequences include 15 participants performing a set of 12 ADLs (e.g., walking, sitting, and jogging). The actions were chosen to present different challenges to BPE algorithms, including different movement speeds, self-occlusions, and complex body poses. Moreover, multi-person sequences, with up to 4 people performing a set of 7 different actions, are provided. Such sequences offer challenging scenarios where multiple self-occluded persons move and interact in a restricted space. They allow assessing the accuracy of multi-person tracking algorithms, focused on maintaining frame-by-frame consistent IDs of each detected person. A total of 13.3h of RGB, depth, and markerless BPE data are present in the dataset, corresponding to over 1,400,000 frames obtained from a calibrated network with 5 RGB-D cameras. The inertial suits, on the other hand, allowed to record 3h of inertial motion capture data, corresponding to a total of over 600,000 frames recorded by each of the 17 IMUs used by every suit.

Human activity recognition and clinical biomechanics are challenging problems in physical telerehabilitation medicine. However, most publicly available datasets on human body movements cannot be used to study both problems in an out-of-the-lab movement acquisition setting. The objective of the VIDIMU dataset is to pave the way towards affordable patient tracking solutions for remote daily life activities recognition and kinematic analysis. The VIDIMU dataset includes 54 healthy young adults that were recorded on video and 16 of them were simultaneously recorded using custom IMUs. For each subject, 13 activities were registered using a low-resolution video camera and five Inertial Measurement Units (IMUs). Inertial sensors were placed in the lower or the upper limbs of the subject, respectively for activities that involve movement with the lower or the upper body. Video recordings were postprocessed using the state-of-the-art pose estimator BodyTrack (similar to OpenPose, and included in NVIDIA Maxine-AR-SDK) to provide a sequence of 3D joint positions for each movement. Raw IMU recordings were post-processed to compute joint angles by inverse kinematics with OpenSim. For recordings including simultaneous acquisition of video and IMU data types, these signals were used for data file synchronization. Collected data can be further used in applications related to human activity recognition and biomechanics related experiments in simulated home-like settings.