Temporal Action Localization

The Gaming 3D Dataset (G3D) focuses on real-time action recognition in a gaming scenario. It contains 10 subjects performing 20 gaming actions: “punch right”, “punch left”, “kick right”, “kick left”, “defend”, “golf swing”, “tennis swing forehand”, “tennis swing backhand”, “tennis serve”, “throw bowling ball”, “aim and fire gun”, “walk”, “run”, “jump”, “climb”, “crouch”, “steer a car”, “wave”, “flap” and “clap”. Source: Skeleton Based Action Recognition Using Translation-Scale Invariant Image Mapping And Multi-Scale Deep CNN Image Source: G3D: A gaming action dataset and real time action recognition evaluation framework

SBU-Kinect-Interaction dataset version 2.0 comprises of RGB-D video sequences of humans performing interaction activities that are recording using the Microsoft Kinect sensor. This dataset was originally recorded for a class project, and it must be used only for the purposes of research. If you use this dataset in your work, please cite the following paper. Kiwon Yun, Jean Honorio, Debaleena Chattopadhyay, Tamara L. Berg, and Dimitris Samaras, The 2nd International Workshop on Human Activity Understanding from 3D Data at Conference on Computer Vision and Pattern Recognition (HAU3D-CVPRW), CVPR 2012 SBU-Refine: SBU-Refine relabels the test set manually and refines the noise labels in training set by algorithm. H. Yang, T. Wang, X. Hu, and C.-W. Fu, “SILT: Shadow-aware iterative label tuning for learning to detect shadows from noisy labels,” in ICCV, 2023, pp. 12 687–12 698.

The Drive&Act dataset is a state of the art multi modal benchmark for driver behavior recognition. The dataset includes 3D skeletons in addition to frame-wise hierarchical labels of 9.6 Million frames captured by 6 different views and 3 modalities (RGB, IR and depth). It offers following key features: * 12h of video data in 29 long sequences * Calibrated multi view camera system with 5 views * Multi modal videos: NIR, Depth and Color data * Markerless motion capture: 3D Body Pose and Head Pose * Model of the static interior of the car * 83 manually annotated hierarchical activity labels: * Level 1: Long running tasks (12) * Level 2: Semantic actions (34) * Level 3: Object Interaction tripplets [action|object|location] (6|17|14) Source: Drive&Act: A Multi-Modal Dataset for Fine-Grained Driver Behavior Recognition in Autonomous Vehicles

A three million frame, multi-view, furniture assembly video dataset that includes depth, atomic actions, object segmentation, and human pose. Source: The IKEA ASM Dataset: Understanding People Assembling Furniture through Actions, Objects and Pose

FineAction contains 103K temporal instances of 106 action categories, annotated in 17K untrimmed videos. FineAction introduces new opportunities and challenges for temporal action localization, thanks to its distinct characteristics of fine action classes with rich diversity, dense annotations of multiple instances, and co-occurring actions of different classes.

A new large-scale dataset for understanding human motions, poses, and actions in a variety of realistic events, especially crowd & complex events. It contains a record number of poses (>1M), the largest number of action labels (>56k) for complex events, and one of the largest number of trajectories lasting for long terms (with average trajectory length >480). Besides, an online evaluation server is built for researchers to evaluate their approaches. Source: Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Events

The dataset collected at the University of Florence during 2012, has been captured using a Kinect camera. It includes 9 activities: wave, drink from a bottle, answer phone,clap, tight lace, sit down, stand up, read watch, bow. During acquisition, 10 subjects were asked to perform the above actions for 2/3 times. This resulted in a total of 215 activity samples. Source: https://www.micc.unifi.it/resources/datasets/florence-3d-actions-dataset/ Image Source: https://www.micc.unifi.it/resources/datasets/florence-3d-actions-dataset/

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

Extreme Pose Interaction (ExPI) Dataset is a new person interaction dataset of Lindy Hop dancing actions. In Lindy Hop, the two dancers are called leader and follower. The authors recorded two couples of dancers in a multi-camera setup equipped also with a motion-capture system. 16 different actions are performed in ExPI dataset, some by the two couples of dancers, some by only one of the couples. Each action was repeated five times to account for variability. More precisely, for each recorded sequence, ExPI provides: (i) Multi-view videos at 25FPS from all the cameras in the recording setup; (ii) Mocap data (3D position of 18 joints for each person) at 25FPS synchronized with the videos.; (iii) camera calibration information; and (iv) 3D shapes as textured meshes for each frame. Overall, the dataset contains 115 sequences with 30k visual frames for each viewpoint and 60k 3D instances annotated

HVU is organized hierarchically in a semantic taxonomy that focuses on multi-label and multi-task video understanding as a comprehensive problem that encompasses the recognition of multiple semantic aspects in the dynamic scene. HVU contains approx.~572k videos in total with 9 million annotations for training, validation, and test set spanning over 3142 labels. HVU encompasses semantic aspects defined on categories of scenes, objects, actions, events, attributes, and concepts which naturally captures the real-world scenarios. Source: Large Scale Holistic Video Understanding

First-Person Hand Action Benchmark is a collection of RGB-D video sequences comprised of more than 100K frames of 45 daily hand action categories, involving 26 different objects in several hand configurations. Source: First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations

We present a comprehensive framework for egocentric interaction recognition using markerless 3D annotations of two hands manipulating objects. To this end, we propose a method to create a unified dataset for egocentric 3D interaction recognition. Our method produces annotations of the 3D pose of two hands and the 6D pose of the manipulated objects, along with their interaction labels for each frame. Our dataset, called H2O (2 Hands and Objects), provides synchronized multi-view RGB-D images, interaction labels, object classes, ground-truth 3D poses for left & right hands, 6D object poses, ground-truth camera poses, object meshes and scene point clouds. To the best of our knowledge, this is the first benchmark that enables the study of first-person actions with the use of the pose of both left and right hands manipulating objects and presents an unprecedented level of detail for egocentric 3D interaction recognition. We further propose the method to predict interaction classes by estimating the 3D pose of two hands and the 6D pose of the manipulated objects, jointly from RGB images. Our method models both inter- and intra-dependencies between both hands and objects by learning the topology of a graph convolutional network that predicts interactions. We show that our method facilitated by this dataset establishes a strong baseline for joint hand-object pose estimation and achieves state-of-the-art accuracy for first person interaction recognition.

The SIND dataset is based on 4K video captured by drones, providing information including traffic participant trajectories, traffic light status, and high-definition maps

MUSES is a large-scale dataset for temporal event (action) localization. It focuses on the temporal localization of multi-shot events, which are captured with multiple shots. Such events often appear in edited videos, such as TV shows and movies. What’s included in MUSES: * 3,697 videos of TV and movie dramas * 716 hours of duration * 25 event categories * 652k shots * 31,477 annotated event instances

Casia V1 is a dataset for forgery classification. Casia V1+ is a modification of the Casia V1 dataset proposed by Chen et al. that replaces authentic images that also exist in Casiav2 with images from the COREL dataset to avoid data contamination.

The GTA Indoor Motion dataset (GTA-IM) that emphasizes human-scene interactions in the indoor environments. It consists of HD RGB-D image sequences of 3D human motion from a realistic game engine. The dataset has clean 3D human pose and camera pose annotations, and large diversity in human appearances, indoor environments, camera views, and human activities. Source: https://github.com/ZheC/GTA-IM-Dataset Image Source: https://github.com/ZheC/GTA-IM-Dataset

Perception Test is a benchmark designed to evaluate the perception and reasoning skills of multimodal models. It introduces real-world videos designed to show perceptually interesting situations and defines multiple tasks that require understanding of memory, abstract patterns, physics, and semantics – across visual, audio, and text modalities. The benchmark consists of 11.6k videos, 23s average length, filmed by around 100 participants worldwide. The videos are densely annotated with six types of labels: object and point tracks, temporal action and sound segments, multiple-choice video question-answers and grounded video question-answers. The benchmark probes pre-trained models for their transfer capabilities, in a zero-shot / few-shot or fine tuning regime.

COVERAGE contains copymove forged (CMFD) images and their originals with similar but genuine objects (SGOs). COVERAGE is designed to highlight and address tamper detection ambiguity of popular methods, caused by self-similarity within natural images. In COVERAGE, forged–original pairs are annotated with (i) the duplicated and forged region masks, and (ii) the tampering factor/similarity metric. For benchmarking, forgery quality is evaluated using (i) computer vision-based methods, and (ii) human detection performance. Source: COVERAGE

Home Action Genome is a large-scale multi-view video database of indoor daily activities. Every activity is captured by synchronized multi-view cameras, including an egocentric view. There are 30 hours of vides with 70 classes of daily activities and 453 classes of atomic actions.

Includes egocentric videos containing hands in the wild. Source: Analysis of Hand Segmentation in the Wild

A self-driving dataset for motion prediction, containing over 1,000 hours of data. This was collected by a fleet of 20 autonomous vehicles along a fixed route in Palo Alto, California, over a four-month period. It consists of 170,000 scenes, where each scene is 25 seconds long and captures the perception output of the self-driving system, which encodes the precise positions and motions of nearby vehicles, cyclists, and pedestrians over time. Source: One Thousand and One Hours: Self-driving Motion Prediction Dataset

This is a 3D action recognition dataset, also known as 3D Action Pairs dataset. The actions in this dataset are selected in pairs such that the two actions of each pair are similar in motion (have similar trajectories) and shape (have similar objects); however, the motion-shape relation is different. Source: HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences

The Spotify Music Streaming Sessions Dataset (MSSD) consists of 160 million streaming sessions with associated user interactions, audio features and metadata describing the tracks streamed during the sessions, and snapshots of the playlists listened to during the sessions. This dataset enables research on important problems including how to model user listening and interaction behaviour in streaming, as well as Music Information Retrieval (MIR), and session-based sequential recommendations.

NTU RGB+D 2D is a curated version of NTU RGB+D often used for skeleton-based action prediction and synthesis. It contains less number of actions.

The TUM Kitchen dataset is an action recognition dataset that contains 20 video sequences captured by 4 cameras with overlapping views. The camera network captures the scene from four viewpoints with 25 fps, and every RGB frame is of the resolution 384×288 by pixels. The action labels are frame-wise, and provided for the left arm, the right arm and the torso separately. Source: Temporal Human Action Segmentation via Dynamic Clustering Image Source: https://ias.in.tum.de/dokuwiki/software/kitchen-activity-data

The BEOID dataset includes object interactions ranging from preparing a coffee to operating a weight lifting machine and opening a door. The dataset is recorded at six locations: kitchen, workspace, laser printer, corridor with a locked door, cardiac gym, and weight-lifting machine. For the first four locations, sequences from five different operators were recorded (two sequences per operator), and from three operators for the last two locations (three sequences per operator). The wearable gaze tracker hardware (ASL Mobile Eye XG) was used to record the dataset. Synchronized wide-lens video data with calibrated 2D gaze fixations are available. Moreover, we release 3D information using a pre-built cloud point map and PTAM tracking. Three-dimensional information of the image and the gaze fixations are included.

A dataset which provides detailed annotations for activity recognition. Source: Recognizing Fine-Grained and Composite Activities using Hand-Centric Features and Script Data

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

WEAR is an outdoor sports dataset for both vision- and inertial-based human activity recognition (HAR). The dataset comprises data from 22 participants performing a total of 18 different workout activities with untrimmed inertial (acceleration) and camera (egocentric video) data recorded at 11 different outside locations. Unlike previous egocentric datasets, WEAR provides a challenging prediction scenario marked by purposely introduced activity variations as well as an overall small information overlap across modalities.

This dataset is an OSN-transmitted (OSN = Online Social Network) version of the CASIA dataset. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (OSN = Online Social Network) version of the CASIA dataset. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (OSN = Online Social Network) version of the CASIA dataset. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (OSN = Online Social Network) version of the CASIA dataset. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the Columbia dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the Columbia dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the Columbia dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.