2D Pose Estimation

The Human3.6M dataset is one of the largest motion capture datasets, which consists of 3.6 million human poses and corresponding images captured by a high-speed motion capture system. There are 4 high-resolution progressive scan cameras to acquire video data at 50 Hz. The dataset contains activities by 11 professional actors in 17 scenarios: discussion, smoking, taking photo, talking on the phone, etc., as well as provides accurate 3D joint positions and high-resolution videos. Source: Space-Time Representation of People Based on 3D Skeletal Data: A Review Image Source: Yu et al

The 300-W is a face dataset that consists of 300 Indoor and 300 Outdoor in-the-wild images. It covers a large variation of identity, expression, illumination conditions, pose, occlusion and face size. The images were downloaded from google.com by making queries such as “party”, “conference”, “protests”, “football” and “celebrities”. Compared to the rest of in-the-wild datasets, the 300-W database contains a larger percentage of partially-occluded images and covers more expressions than the common “neutral” or “smile”, such as “surprise” or “scream”. Images were annotated with the 68-point mark-up using a semi-automatic methodology. The images of the database were carefully selected so that they represent a characteristic sample of challenging but natural face instances under totally unconstrained conditions. Thus, methods that achieve accurate performance on the 300-W database can demonstrate the same accuracy in most realistic cases. Many images of the database contain more than one annotated faces (293 images with 1 face, 53 images with 2 faces and 53 images with [3, 7] faces). Consequently, the database consists of 600 annotated face instances, but 399 unique images. Finally, there is a large variety of face sizes. Specifically, 49.3% of the faces have size in the range [48.6k, 2.0M] and the overall mean size is 85k (about 292 × 292) pixels. Source: https://ibug.doc.ic.ac.uk/media/uploads/documents/sagonas_2016_imavis.pdf Image Source: https://www.researchgate.net/profile/Xuanyi_Dong/publication/323722412/figure/fig1/AS:679426136227845@1538999222829/Face-samples-from-300-W-dataset-Different-faces-have-different-styles-whereas-the-style_Q640.jpg

Animal Kingdom is a large and diverse dataset that provides multiple annotated tasks to enable a more thorough understanding of natural animal behaviors. The wild animal footage used in the dataset records different times of the day in an extensive range of environments containing variations in backgrounds, viewpoints, illumination and weather conditions. More specifically, the dataset contains 50 hours of annotated videos to localize relevant animal behavior segments in long videos for the video grounding task, 30K video sequences for the fine-grained multi-label action recognition task, and 33K frames for the pose estimation task, which correspond to a diverse range of animals with 850 species across 6 major animal classes.

The first large-scale pose dataset containing objects of multiple super-categories, termed Multi-category Pose (MP-100). In total, MP-100 dataset covers 100 subcategories and 8 super-categories. Over 18K images and 20K annotations are collected from several popular 2D pose datasets, including COCO, 300W, AFLW, OneHand10K, DeepFasion2, AP-10K, MacaquePose, Vinegar Fly, Desert Locust, CUB-200, CarFusion, AnimalWeb and Keypoint-5

We introduce HARPER, a novel dataset for 3D body pose estimation and forecast in dyadic interactions between users and \spot, the quadruped robot manufactured by Boston Dynamics. The key-novelty is the focus on the robot's perspective, i.e., on the data captured by the robot's sensors. These make 3D body pose analysis challenging because being close to the ground captures humans only partially. The scenario underlying HARPER includes 15 actions, of which 10 involve physical contact between the robot and users. The Corpus contains not only the recordings of the built-in stereo cameras of Spot, but also those of a 6-camera OptiTrack system (all recordings are synchronized). This leads to ground-truth skeletal representations with a precision lower than a millimeter. In addition, the Corpus includes reproducible benchmarks on 3D Human Pose Estimation, Human Pose Forecasting, and Collision Prediction, all based on publicly available baseline approaches. This enables future HARPER users to rigorously compare their results with those we provide in this work. Source: [Download dataset] (https://github.com/intelligolabs/HARPER)

The dataset is designed specifically to solve a range of computer vision problems (2D-3D tracking, posture) faced by biologists while designing behavior studies with animals. Typically, datasets for animal-specific vision tasks are created using open-source video material. This might be effective for an initial start, but these methods are not deployment ready for the behavior community. Therefore, we designed a semi-automated method for biologists to create well-curated datasets at a large scale for the ML and Vision community. 3D-POP is the first dataset with 3D ground truth for multi-animal, multi-view tracking problems. Highlight: The dataset is captured with the intention of using it for various vision problems and with different levels of complexity (no of cameras, no of individuals) Video explanation: Link to YouTube video Video teaser: Link to YouTube video ## Dataset Features: ### Marker-based videos: - 6 hours+ of annotations of 18 individuals (groups of 1, 2, 5, 10). - Bounding box - Trajectories (2D and 3D) - Posture (2D and 3D) with 9 key points - Identities - Total of 57 sequences (4K) with 4 views. - Dataset customization* (Users can modify the dataset and add key points to the dataset) ### Markerless: - 1Hr+ videos of 18 individuals in groups of 1, 2, 5, 11. The birds have no markers. This data is provided as test cases and unsupervised approaches. ## Problems: ### 2D domain: - Position, Posture of birds (different group sizes n = 1, 2, 5, 10) with Single/Multiview. - Tracking with single - multiview ### 3D domain: - Position, Posture of birds (different group sizes n = 1, 2, 5, 10) with Single/Multiview. - Tracking with single - multiview ### Fine-grained recognition: - Identity tracking with ground truth. ### Unsupervised learning: - 2D or 3D posture problems ## Idea: The dataset is created with a motion capture system, using the 6-DOF tracking ability. Assumptions are that head and body act as rigid bodies when birds walk and forage (proved with experiment). Therefore, we get the 3D position of key points by tracking head/body orientation.

Description: The "iRodent" dataset contains rodent species observations obtained using the iNaturalist API, with a focus on Suborder Myomorpha (Taxon ID: 16). The dataset features prominent rodent species like Muskrat, Brown Rat, House Mouse, Black Rat, Hispid Cotton Rat, Meadow Vole, Bank Vole, Deer Mouse, White-footed Mouse, and Striped Field Mouse. The dataset provides manually labeled keypoints for pose estimation and segmentation masks for a subset of images using a Mask R-CNN model. If you use this data, please cite: Ye, S., Filippova, A., Lauer, J. et al. SuperAnimal pretrained pose estimation models for behavioral analysis. Nat Commun 15, 5165 (2024). https://doi.org/10.1038/s41467-024-48792-2 Creator: Adaptive Intelligence Lab at EPFL: https://www.mackenziemathislab.org/ Data Format: COCO format Number of Images: 443 Species: Muskrat, Brown Rat, House Mouse, Black Rat, Hispid Cotton Rat, Meadow Vole, Bank Vole, Deer Mouse, White-footed Mouse, Striped Field Mouse Image Resolution: Varied (800x600 to 5184x3456 pixels) Annotations: Pose keypoints and generated segmentation masks by Tian Qiu and Mackenzie Mathis. License: Apache 2.0 Keywords: animal pose estimation, behaviour analysis, keypoints, rodent Contact: Mackenzie Mathis Email: mackenzie.mathis@epfl.ch

This dataset comprehends the 3D building information model (in IFC and Revit formats), manually elaborated based on the terrestrial laser scanner of the sequence 2 of ConSLAM, and the refined ground truth (GT) poses (in TUM format) of sessions 2, 3, 4, and 5 of the open-access ConSLAM dataset (which provides camera, LiDAR, and IMU measurements). This dataset enables the evaluation of pose estimation algorithms, such as localization, odometry, SLAM, and Bundle adjustment. Given the differences between the real-world and the 3D reference BIM model (Scan-Map deviations), the dataset is appropriate for evaluating long-term localization and mapping capabilities, i.e., methods that are robust to changes in the environment. In particular, in a real-world construction environment with normal clutter (not as in the Hilti dataset).

#Introduction This dataset supports Ye et al. 2024 Nature Communications (https://www.nature.com/articles/s41467-024-48792-2). Ye, S., Filippova, A., Lauer, J. et al. SuperAnimal pretrained pose estimation models for behavioral analysis. Nat Commun 15, 5165 (2024). https://doi.org/10.1038/s41467-024-48792-2 Please cite this dataset and paper if you use this resource. Please also see Ye et al. 2024 for the full DataSheet that accompanies this download, including the meta data for how to use this data is you want to compare model results on benchmark tasks. Below is just a summary. Also see the dataset licensing below. #Data SuperAnimal-TopViewMouse-5K includes the following datasets: - CSI, BM, EPM, LDB, OFT See full details at (1) and in (2). - BlackMice See full details at (3). - WhiteMice Courtesy of Prof. Sam Golden and Nastacia Goodwin. See details in SIMBA (4). TriMouse See full details at (5). - DLC-Openfield See full details at (6). - Kiehn-Lab-Openfield, Swimming, and treadmill Courtesy of Prof. Ole Kiehn, Dr. Jared Cregg, and Prof. Carmelo Bellardita; see details at (7). - MausHaus We collected video data from five single-housed C57BL/6J male and female mice in an extended home cage, carried out in the laboratory of Mackenzie Mathis at Harvard University and also EPFL (temperature of housing was 20-25C, humidity 20-50%). Data were recorded at 30Hz with 640 × 480 pixels resolution acquired with White Matter, LLC eV cameras. Annotators localized 26 keypoints across 322 frames sampled from within DeepLabCut using the k-means clustering approach (8). All experimental procedures for mice were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and approved by the Harvard Institutional Animal Care and Use Committee (IACUC) (n=1 mouse), and by the Veterinary Office of the Canton of Geneva (Switzerland; license GE01) (n=4 mice). Here is an image with examples from the datasets, the distribution of images per dataset, and the keypoint guide. #Ethical Considerations • Data was collected with IUCAC or other governmental approval. Each individual dataset used in training reports the ethics approval they obtained. #Caveats and Recommendations • Please note that each training dataset was labeled by separate labs and different individuals, therefore while we map names to a unified pose vocabulary, there will be annotator bias in keypoint placement (See Ye et al. 2024 for our Supplementary Note on annotator bias). • Note the dataset is primarily using C56Blk6/J mice and only some CD1 examples. #License Modified MIT. Copyright 2023-present by Mackenzie Mathis, Shaokai Ye, and contributors. Permission is hereby granted to you (hereafter "LICENSEE") a fully-paid, non-exclusive, and non-transferable license for academic, non-commercial purposes only (hereafter “LICENSE”) to use the "DATASET" subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software: This data or resulting software may not be used to harm any animal deliberately. LICENSEE acknowledges that the DATASET is a research tool. THE DATASET IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE DATASET OR THE USE OR OTHER DEALINGS IN THE DATASET. If this license is not appropriate for your application, please contact Prof. Mackenzie W. Mathis (mackenzie@post.harvard.edu) for a commercial use license. Please cite Ye et al if you use this DATASET in your work. #References 1. Oliver Sturman, Lukas von Ziegler, Christa Schläppi, Furkan Akyol, Mattia Privitera, Daria Slominski, Christina Grimm, Laetitia Thieren, Valerio Zerbi, Benjamin Grewe, et al. Deep learning-based behavioral analysis reaches human accuracy and is capable of outperforming commercial solutions. Neuropsychopharmacology, 45(11):1942–1952, 2020. 2. Lukas von Ziegler, Oliver Sturman, and Johannes Bohacek. Videos for deeplabcut, noldus ethovision X14 and TSE multi conditioning systems comparisons. https://doi.org/10.5281/zenodo.3608658. Zenodo, January 2020. 3. Isaac Chang. Trained DeepLabCut model for tracking mouse in open field arena with topdown view. https://doi.org/10.5281/zenodo.3955216. Zenodo, July 2020. 4. Simon RO Nilsson, Nastacia L. Goodwin, Jia Jie Choong, Sophia Hwang, Hayden R Wright, Zane C Norville, Xiaoyu Tong, Dayu Lin, Brandon S. Bentzley, Neir Eshel, Ryan J McLaughlin, and Sam A. Golden. Simple behavioral analysis (simba) – an open source toolkit for computer classification of complex social behaviors in experimental animals. bioRxiv, 2020. 5. Jessy Lauer, Mu Zhou, Shaokai Ye, William Menegas, Steffen Schneider, Tanmay Nath, Mohammed Mostafizur Rahman, Valentina Di Santo, Daniel Soberanes, Guoping Feng, Venkatesh N. Murthy, George Lauder, Catherine Dulac, Mackenzie W. Mathis, and Alexander Mathis. Multi-animal pose estimation, identification and tracking with deeplabcut. Nature Methods, 19:496 – 504, 2022. 6. Alexander Mathis, Pranav Mamidanna, Kevin M Cury, Taiga Abe, Venkatesh N Murthy, Mackenzie Weygandt Mathis, and Matthias Bethge. Deeplabcut: markerless pose estimation of user-defined body parts with deep learning. Nature neuroscience, 21:1281–1289, 2018. 7. Jared M. Cregg, Roberto Leiras, Alexia Montalant, Paulina Wanken, Ian R. Wickersham, and Ole Kiehn. Brainstem neurons that command mammalian locomotor asymmetries. Nature neuroscience, 23:730 – 740, 2020

To file a complaint with Expedia, the first step is to dial ((＋1 (888) 829-0881)) (USA). Once you're connected to an Expedia human, explain any concerns related to refund fees, hotel cancellations, or cancellation rates. If you need help understanding the Expedia hotel refund policy, their team can provide clarity. For persistent issues, call ((＋1 (888) 829-0881)) (USA) and escalate your complaint to a higher level. Expedia is committed to resolving your complaint, and you can reach them again at ((＋1 (888) 829-0881)) (USA) to ensure your matter is taken care of. How do I file a complaint with Expedia? If you need to complain about an Expedia booking, start by calling ((＋1 (888) 829-0881)) (USA). Speak to an Expedia human about any issues with your booking, including hotel cancellations, refund rates, or the Expedia hotel refund policy. If you’re dealing with an Expedia cancellation rate or refund fee, customer service can clarify all related concerns. Don’t forget that if you need additional assistance, you can reach out to ((＋1 (888) 829-0881)) (USA) to resolve your issue and ensure your complaint is addressed appropriately. 1. Call Expedia Support Directly The most direct way to file a complaint is to call Expedia's customer service at 1-(888) 829-0881. This number connects you with their support team, where you can explain your issue clearly and request assistance. Make sure to have the following ready when you call: If you prefer to file your complaint online, visit the Expedia Help Center. Log in to your account and go to the section related to your issue—like cancellations, refunds, or booking errors. You can then submit a formal complaint or request a callback. 4. Post on Social Media Sometimes, reaching out on platforms like Twitter or Facebook can draw attention to your case. Tag @Expedia and briefly explain your issue. Companies often respond faster when complaints are public. Still, it’s best to combine this approach with a phone call to 1-(888) 829-0881 for proper tracking. 5. Escalate to Consumer Protection Agencies If Expedia doesn’t resolve your complaint, you can file with external agencies such as: BBB (Better Business Bureau) FTC (Federal Trade Commission) Filing a complaint with Expedia doesn’t have to be stressful. Start by calling 1-(888) 829-0881, stay calm and organized, and follow up if necessary. Use your resources—like social media and consumer agencies—if your issue goes unresolved. With persistence and clear communication, you’ll improve your chances of getting a solution.

ConSLAM is a real-world dataset collected periodically on a construction site to measure the accuracy of mobile scanners' SLAM algorithms. The dataset contains time-synchronized and spatially registered RGB and NIR images and 360-deg LiDAR scans, 9-axis IMU measurements, and professional ground-truth terrestrial laser scans. This dataset reflects the periodic need to scan construction sites with the aim of accurately monitoring progress using a hand-held scanner. The sensors used for data acquisition are: - LiDAR: Velodyne VLP-16. - RGB camera: Alvium U-319c, 3.2 MP. - NIR camera: Alvium 1800 U-501, 5.0 MP. - 9-axis IMU: Xsens MTi-610.

Daily Activity Recordings for Artificial Intelligence (DARai, pronounced "Dahr-ree") is a multimodal, hierarchically annotated dataset constructed to understand human activities in real-world settings. DARai consists of continuous scripted and unscripted recordings of 50 participants in 10 different environments, totaling over 200 hours of data from 20 sensors including multiple camera views, depth and radar sensors, wearable inertial measurement units (IMUs), electromyography (EMG), insole pressure sensors, biomonitor sensors, and gaze tracker. To capture the complexity in human activities, DARai is annotated at three levels of hierarchy: (i) high-level activities (L1) that are independent tasks, (ii) lower-level actions (L2) that are patterns shared between activities, and (iii) fine-grained procedures (L3) that detail the exact execution steps for actions. The dataset annotations and recordings are designed so that 22.7% of L2 actions are shared between L1 activities and 14.2% of L3 procedures are shared between L2 actions. The overlap and unscripted nature of DARai allows counterfactual activities in the dataset. Experiments with various machine learning models showcase the value of DARai in uncovering important challenges in human-centered applications. Specifically, we conduct unimodal and multimodal sensor fusion experiments for recognition, temporal localization, and future action anticipation across all hierarchical annotation levels. To highlight the limitations of individual sensors, we also conduct domain-variant experiments that are enabled by DARai's multi-sensor and counterfactual activity design setup. The code, documentation, and dataset are available at the dedicated DARai website.

Desert Locus is a animal pose estimation dataset for desert locuses.

MacaquePose is an animal pose estimation dataset containing pictures of macaque monkeys and manually labeled annotations on them.

Vinegar Fly is a pose estimation dataset for fruit flies.