A large-scale hand pose dataset, collected using a novel capture method. Source: BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis

The CIP dataset is composed of 2 subsets, containing low-cost (MPU9250) and high-end (MTwAwinda) Magnetic, Angular Rate, and Gravity (MARG) sensor data respectively. It provides data for the analysis of the complete inertial pose pipeline, from raw measurements, to sensor-to-segment calibration, multi-sensor fusion, skeleton kinematics, to the complete human pose. Multiple trials were collected with 21 and 10 subjects respectively, performing 6 types of movements (ranging from calibration, to daily-activities, range-of-motion and random). It presents a high degree of variability and complex dynamics while containing common sources of error found on real conditions. This amounts to 3.5M samples, synchronized with a ground-truth inertial motion capture system (Xsens) at 60hz. This dataset may contribute to assess, benchmark and develop novel algorithms for each of the pipelines' processing steps, with applications in classic or data-driven inertial pose estimation algorithms, human movement understanding and forecasting and ergonomic assessment in industrial or rehabilitation settings.

COCO-MEBOW (Monocular Estimation of Body Orientation in the Wild) is a new large-scale dataset for orientation estimation from a single in-the-wild image. The body-orientation labels for 133380 human bodies within 55K images from the COCO dataset have been collected using an efficient and high-precision annotation pipeline. There are 127844 human instance in training set and 5536 human instance in validation set.

This dataset derives from Coil100. There are more than 1,1M images of 100 objects. Each object was turned on a turnable through 360 degrees to vary object pose with respect to a fixed color camera. Images of the objects were taken at pose intervals of 5 degrees. This corresponds to 72 poses per object. Then planar rotation (9 angles) and 18 scaling factors has been applied. Objects have a wide variety of complex geometric and reflectance characteristics. This augmented version of Coil100 has been design for Disentangled Representation Learning for real images, the Factors of Variations are: | Factors | # values | |----------|----------| | Object | 100 | | 3D Pose | 72 | | Rotation | 9 | | Scale | 18 | The binarized version is also available.

ConSLAM is a real-world dataset collected periodically on a construction site to measure the accuracy of mobile scanners' SLAM algorithms. The dataset contains time-synchronized and spatially registered RGB and NIR images and 360-deg LiDAR scans, 9-axis IMU measurements, and professional ground-truth terrestrial laser scans. This dataset reflects the periodic need to scan construction sites with the aim of accurately monitoring progress using a hand-held scanner. The sensors used for data acquisition are: - LiDAR: Velodyne VLP-16. - RGB camera: Alvium U-319c, 3.2 MP. - NIR camera: Alvium 1800 U-501, 5.0 MP. - 9-axis IMU: Xsens MTi-610.

DynOPETs is a real-world RGB-D dataset designed for object pose estimation and tracking in dynamic scenes with moving cameras. COPE119 119 sequences covering 6 common categories from the COPE benchmark: bottles, bowls, cameras, cans, laptops, mugs. Designed for COPE (Category-level Pose Estimation) methods.

A dataset with 3200 images (200 for each number quantity on each hand).

Data for "Image-based Backbone Reconstruction for Non-Slender Soft Robots" This dataset provides the data for the forthcoming paper "Image-based Backbone Reconstruction for Non-Slender Soft Robots". The backbone reconstruction method used is based on the method described in Hoffmann et al. [1]. The modifications to this method to support the non-slender soft robot in this dataset are described in the forthcoming paper mentioned above. This dataset holds raw images of pressurized and elongated soft robots and the corresponding reconstructed backbones. ## Dataset The dataset is split into two subsets with similar structure. The first subset is contained in dataset_01. The second dataset is contained in dataset_02. Each subset consists of five folders and one schedule file. The schedule file schedule.csv contains the index of the schedule entry, the angle $\alpha$ in degree, the pressure of each chamber $p_1$ to $p_3$ in bar and if the pressurization is active. Furthermore, the five folders of the subset can be described as follows - raw: Contains the raw cropped images. The filenames are formatted as CROPPED_C{CAMERA_INDEX}_E{SCHEDULE_ENTRY}.png with the camera index CAMERA_INDEX and the schedule entry SCHEDULE_ENTRY. - constant_curvature_slender, constant_curvature_volumetric, cubic_curvature_slender and cubic_curvature_volumetric. These folders contain the actual reconstructed backbones based on the raw data from the raw folder. A different reconstruction approach was used in each of these folders - constant_curvature_slender - A constant curvature backbone kinematic based on the slender model, - constant_curvature_volumetric - A constant curvature backbone kinematic based on the volumetric model, - cubic_curvature_slender - A cubic curvature backbone kinematic based on the slender model, - cubic_curvature_volumetric - A cubic curvature backbone kinematic based on the volumetric model. Each of these folders contain a data and figures folder. The data folder consists of PARAMETER_E{SCHEDULE_ENTRY}.json files listing the optimization parameters for each schedule entry SCHEDULE_ENTRY in the JSON format. The figures folder contains annotated images of the reconstructed backbone on the cropped raw images. The filenames are structured ANNOTATED_E{SCHEDULE_ENTRY}_C{CAMERA_INDEX}_EPOCH{EPOCH}.png with the schedule entry SCHEDULE_ENTRY, the camera index CAMERA_INDEX and the epoch EPOCH of the optimization algorithm. The optimization parameters include the base position base_position of the reconstructed backbone in world coordinates, the coefficients for the curvature polynomials ux and uy, and the constant coefficient for the elongation polynomial la. ## Calibration Data The calibration data is located in the calibration folder and consists of multiple .npy files in the numpy format. The corresponding camera index for the calibrated camera is abbreviated with CAMERA_INDEX in the following: - C{CAMERA_INDEX}.npy - Stores the reprojection error, camera matrix, distortion coefficients, rotation, and translation vectors as returned by the cv2.calibrateCamera [2] method. - C{CAMERA_INDEX}_camera_matrix.npy - Stores the camera_matrix as returned by the cv2.calibrateCamera [2] method. - C{CAMERA_INDEX}_distortion_coefficients.npy - Stores the distortion coefficients as returned by the cv2.calibrateCamera [2] method. - C{CAMERA_INDEX}_projection_matrix.npy - Stores the projection matrix from world space to pixel space based on the stereo camera calibration. - STEREO.npy - Stores the reprojection error, R, T, E, F as returned by the cv2.stereoCalibrate [2] method as an object datatype. ## Acknowledgement Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – 501861263 – SPP2353 ## References [1] M. K. Hoffmann, J. Mühlenhoff, Z. Ding, T. Sattel and K. Flaßkamp. An iterative closest point algorithm for marker-free 3D shape registration of continuum robots. arXiv. https://arxiv.org/abs/2405.15336 [2] OpenCV. Camera Calibration and 3D Reconstruction. OpenCV Documentation. https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html, accessed May 27, 2024.

DensePose-Track is a dataset of videos where selected frames are annotated in the traditional DensePose manner. Source: Slim DensePose: Thrifty Learning from Sparse Annotations and Motion Cues

Desert Locus is a animal pose estimation dataset for desert locuses.

EgoPW training dataset with scene annotations Since we want to generalize to data captured with a real head-mounted camera, we also extended the EgoPW training dataset. For this, we first reconstruct the scene geometry from the egocentric image sequences of the EgoPW training dataset with a Structure-from-Motion (SfM) algorithm. This step provides a dense reconstruction of the background scene. The global scale of the reconstruction is recovered from known objects present in the sequences, such as laptops and chairs. We further render the depth maps of the scene in the egocentric perspective based on the reconstructed geometry. Our EgoPW-Scene dataset contains 92 K frames in total, which are distributed in 30 sequences performed by 5 actors.

A new large-scale dataset that consists of 409 fine-grained categories and 31,881 images with accurate 3D pose annotation. Source: Improving Annotation for 3D Pose Dataset of Fine-Grained Object Categories

The researchers collected 3,500 images of Tilapia fish, with each image containing three fish in a small bowl. These images were manually annotated using Roboflow, a tool for creating and managing annotated datasets. Four keypoints were labeled on each fish: mouth, peduncle, belly, and back. While the primary goal was to measure fish length using the mouth and peduncle points, the additional keypoints (belly and back) were included to support potential future research, such as using girth to determine fish weight. This dataset was used to train a YOLOv8 model for keypoint detection, achieving high accuracy in identifying these crucial points on the Tilapia fish.

FreeMan is the first large-scale multi-view human motion dataset under real scenarios. FreeMan was captured by synchro- nizing 8 smartphones across diverse scenarios. It comprises 11M frames from 8000 sequences, viewed from different perspectives. These sequences cover 40 subjects across 10 different scenarios, each with varying lighting conditions.

GraspClutter6D is a large-scale real-world dataset for robust object perception and robotic grasping in cluttered environments. It features 1,000 highly cluttered scenes with dense arrangements (average 14.1 objects/scene with 62.6% occlusion), 200 household, industrial, and warehouse objects captured in 75 diverse environment configurations (bins, shelves, and tables), multi-view data from 4 RGB-D cameras (RealSense D415, D435, Azure Kinect, and Zivid One+), and comprehensive annotations including 736K 6D object poses and 9.3 billion feasible robotic grasps for 52K RGB-D images. The dataset provides a challenging testbed for segmentation, 6D pose estimation, and grasp detection algorithms in realistic cluttered scenarios.

Halpe-FullBody is a full body keypoints dataset where each person has annotated 136 keypoints, including 20 for body, 6 for feet, 42 for hands and 68 for face. It is designed for the task of whole body human pose estimation. Source: AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time Image Source: https://arxiv.org/pdf/2211.03375v1.pdf

The HumanoidRobotPose dataset is a dataset for real-time pose estimation of humanoid robots.

Consists of 10,000 images is constructed, in which all the immediacy measures and the human poses are annotated. Source: Multi-Task Recurrent Neural Network for Immediacy Prediction

A dataset that consists of 20 actions of various actors, such as tennis serves, yoga and Tai Chi. Take tennis serves as an example. The publicly available videos of some tennis players from YouTube are downloaded, and manually crop the videos roughly to obtain a set of video clips of serves for each player. Source: Motion Capture from Internet Videos

This data set contains over 600GB of multimodal data from a Mars analog mission, including accurate 6DoF outdoor ground truth, indoor-outdoor transitions with continuous cross-domain ground truth, and indoor data with Optitrack measurements as ground truth. With 26 flights and a combined distance of 2.5km, this data set provides you with various distinct challenges for testing and proofing your algorithms. The UAV carries 18 sensors, including a high-resolution navigation camera and a stereo camera with an overlapping field of view, two RTK GNSS sensors with centimeter accuracy, as well as three IMUs, placed at strategic locations: Hardware dampened at the center, off-center with a lever arm, and a 1kHz IMU rigidly attached to the UAV (in case you want to work with unfiltered data). The sensors are fully pre-calibrated, and the data set is ready to use. However, if you want to use your own calibration algorithms, then the raw calibration data is also ready for download. The cross-domain outdoor to indoor transition segments are especially challenging because of realistic sensor behavior such as GNSS degradation and dropouts, changes in the measured magnetic field, and changing light conditions when transitioning to the indoor environment. The data set also provides 4 hours of static sensor data and vibration data with accurate RPM measurements to provide you with additional information about sensor properties and vehicle integrity. The data set provides you with everything you need to test your research: First, you can test your algorithms with the indoor data sets in a controlled environment and then switch to the more challenging flight scenario, such as the transition data, which requires sensor switching, or the Mars analog data with higher velocities, multiple touchdowns, challenging ground structures or constant velocity segments. The Mars analog data also contains cliff flight overs and traverses while the stereo camera faces the cliff in case you want to perform a 3D reconstruction or challenge your SLAM algorithm. The critical aspects of each data set are shown on the website (https://sst.aau.at/cns/datasets), making it easy to find the best data to test or challenge your algorithm.

KITTI-6DoF is a dataset that contains annotations for the 6DoF estimation task for 5 object categories on 7,481 frames. Source: Simultaneous Multiple Object Detection and Pose Estimation using 3D Model Infusion with Monocular Vision Image Source: https://arxiv.org/pdf/2211.11188v1.pdf

MacaquePose is an animal pose estimation dataset containing pictures of macaque monkeys and manually labeled annotations on them.

Human pose estimation (HPE) in the top-view using fisheye cameras presents a promising and innovative application domain. However, the availability of datasets capturing this viewpoint is extremely limited, especially those with high-quality 2D and 3D keypoint annotations. Addressing this gap, we leverage the capabilities of Neural Radiance Fields (NeRF) technique to establish a comprehensive pipeline for generating human pose datasets from existing 2D and 3D datasets, specifically tailored for the top-view fisheye perspective. Through this pipeline, we create a novel dataset NToP (NeRF-powered Top-view human Pose dataset for fisheye cameras) with over 570 thousand images, and conduct an extensive evaluation of its efficacy in enhancing neural networks for 2D and 3D top-view human pose estimation. Extensive evaluations on existing top-view 2D and 3D HPE datasets as well as our new real-world top-view 2D HPE dataset OmniLab prove that our dataset is effective and exceeds previous datasets in this field of research.

In order to evaluate the effectiveness of NToP in real-world scenarios, we collect a new dataset OmniLab with a top-view omnidirectional camera, mounted on the ceiling of two different rooms (bedroom, living room) at 2.5 m height. Five actors (3 males, 2 females) perform 15 actions from CMU-MoCap database (brooming, cleaning windows, down and get up, drinking, fall-on-face, in chair and stand up, pull object, push object, rugpull, turn left, turn right, upbend from knees, upbend from waist, up from ground, walk, walk-old-man) in two rooms with varying clothes. The recorded action length is 2.5 s, which results in 60 images for each scene at a frame rate of 24 FPS. The position of the camera is fixed and the resolution of the images is 1200 by 1200 pixels. A total of 4800 frames are collected. All annotations of 17 keypoints conforming to COCO conventions are estimated through a keypoint detector and subsequently refined by four different humans in two loops to ensure high annotation quality. Bottom figure shows a few examples from OmniLab with person bounding boxes and keypoint annotations.

A multimodal dataset of radio galaxies and their corresponding infrared hosts.

Automating the creation of catalogues for radio galaxies in next-generation deep surveys necessitates the identification of components within extended sources and their respective infrared hosts. We present RadioGalaxyNET, a multimodal dataset, tailored for machine learning tasks to streamline the automated detection and localization of multi-component extended radio galaxies and their associated infrared hosts. The dataset encompasses 4,155 instances of galaxies across 2,800 images, incorporating both radio and infrared channels. Each instance furnishes details about the extended radio galaxy class, a bounding box covering all components, a pixel-level segmentation mask, and the keypoint position of the corresponding infrared host galaxy. RadioGalaxyNET is the first dataset to include images from the highly sensitive Australian Square Kilometre Array Pathfinder (ASKAP) radio telescope, corresponding infrared images, and instance-level annotations for galaxy detection.

Includes several sets of synthetic stereo images labelled with grasp rectangles representing parallel-jaw grasps (Cornell-like format). The set was introduced in the range of the ICAR paper "Automatic generation of realistic training data for learning parallel-jaw grasping from synthetic stereo images", please refer to it if you are using the data.

This is the official dataset collected for RPMArt to test the sim-to-real transfer. It contains 6 articulated object instances, each captured from 20 camera views under 5 states in scenarios with and without background, as well as presence or absence of distractors. All data are collected by a wrist-mounted Intel RealSense L515 LiDAR camera on a 7-DOF Franka Emika robot arm. The details of collection process are presented in the paper.

Synthetic humans generated by the RePoGen method.

SAT-MTB-VSR is a large-scale dataset for satellite video super-resolution made from original videos of Jilin-1, which is a subset of the satellite video multitasking dataset SAT-MTB. The dataset is cropped from 18 videos captured by the Jilin-1 video satellite, covering a wide range of terrains, such as cities, docks, airports, suburbs, forests, and deserts, with a resolution of about 1 m. And the videos contain dynamic scenes, such as moving cars, airplanes, trains, and ships, which test the ability of the VSR method to deal with moving targets of different sizes and speeds. At the same time, due to the motion of the satellite, the video contains changes in viewing angle and lighting. It contains 431 videos, each of which is 100 consecutive frames, of which 413 are used as the training set and 18 as the validation set, and all the 18 validation sets are from different original videos, while the size of the images is 640 × 640. These images are downsampled 4× by bicubic interpolation to get 160 × 160 low-resolution images, thus obtaining the LR-HR training pairs.

SIDOD is a new, publicly-available image dataset generated by the NVIDIA Deep Learning Data Synthesizer intended for use in object detection, pose estimation, and tracking applications. This dataset contains 144k stereo image pairs that synthetically combine 18 camera viewpoints of three photorealistic virtual environments with up to 10 objects (chosen randomly from the 21 object models of the YCB dataset) and flying distractors. Source: SIDOD: A Synthetic Image Dataset for 3D Object Pose Recognition with Distractors Image Source: https://research.nvidia.com/publication/2019-06_SIDOD%3A-A-Synthetic

The SMOT dataset, Single sequence-Multi Objects Training, is collected to represent a practical scenario of collecting training images of new objects in the real world, i.e. a mobile robot with an RGB-D camera collects a sequence of frames while driving around a table to learning multiple objects and tries to recognize objects in different locations. Source: SMOT

3D hand pose data set created using stereo camera - contains 18,000 RGB images and paired depth images - 3D positions of hand joints (21 joints)

The Store Dataset is a dataset for estimating 3D poses of multiple humans in real-time. It is captured inside two kinds of simulated stores with 12 and 28 cameras, respectively. Source: https://arxiv.org/abs/2003.03972

A large-scale dataset composed of object-centric street view scenes along with point correspondences and camera pose information. Source: Generic 3D Representation via Pose Estimation and Matching

Surgical Hands is a dataset that provides multi-instance articulated hand pose annotations for in-vivo videos. The dataset contains 76 video clips from 28 publicly available surgical videos and over 8.1k annotated hand pose instances.