Action Recognition

The AI City Challenge, hosted at CVPR 2024, focuses on harnessing AI to enhance operational efficiency in physical settings such as retail and warehouse environments, and Intelligent Traffic Systems (ITS). It aims to utilize AI for actionable insights from sensor data, like camera feeds, to improve traffic safety and transportation outcomes. This year, the challenge spotlights two key areas with significant potential: retail business and ITS.

TAPOS is a new dataset developed on sport videos with manual annotations of sub-actions, and conduct a study on temporal action parsing on top. A sport activity usually consists of multiple sub-actions and that the awareness of such temporal structures is beneficial to action recognition. TAPOS contains 16,294 valid instances in total, across 21 action classes. These instances have a duration of 9.4 seconds on average. The number of instances within each class is different, where the largest class high jump has over 1,600 instances, and the smallest class beam has 200 instances. The average number of sub-actions also varies from class to class, where parallel bars has 9 sub-actions on average, and long jump has 3 sub-actions on average. All instances are split into train, validation and test sets, of sizes 13094, 1790, and 1763, respectively.

COVERAGE contains copymove forged (CMFD) images and their originals with similar but genuine objects (SGOs). COVERAGE is designed to highlight and address tamper detection ambiguity of popular methods, caused by self-similarity within natural images. In COVERAGE, forged–original pairs are annotated with (i) the duplicated and forged region masks, and (ii) the tampering factor/similarity metric. For benchmarking, forgery quality is evaluated using (i) computer vision-based methods, and (ii) human detection performance. Source: COVERAGE

Home Action Genome is a large-scale multi-view video database of indoor daily activities. Every activity is captured by synchronized multi-view cameras, including an egocentric view. There are 30 hours of vides with 70 classes of daily activities and 453 classes of atomic actions.

We describe the 2020 edition of the DeepMind Kinetics human action dataset, which replenishes and extends the Kinetics-700 dataset. In this new version, there are at least 700 video clips from different YouTube videos for each of the 700 classes. This paper details the changes introduced for this new release of the dataset and includes a comprehensive set of statistics as well as baseline results using the I3D network.

PSI-AVA is a dataset designed for holistic surgical scene understanding. It contains approximately 20.45 hours of the surgical procedure performed by three expert surgeons and annotations for both long-term (Phase and Step recognition) and short-term reasoning (Instrument detection and novel Atomic Action recognition) in robot-assisted radical prostatectomy videos. Source: Towards Holistic Surgical Scene Understanding Image Source: https://arxiv.org/pdf/2212.04582v2.pdf

Kinetics-100 is a dataset split created from the Kinetics dataset to evaluate the performance of few-shot action recognition models. 100 classes are randomly selected from a total of 400 categories, each composed of 100 examples. The 100 classes are further split into 64, 12, and 24 non-overlapping classes to use as the meta-training set, meta-validation set, and meta-testing set, respectively. Link to the selected samples can be found here: https://github.com/ffmpbgrnn/CMN/tree/master/kinetics-100

This is a 3D action recognition dataset, also known as 3D Action Pairs dataset. The actions in this dataset are selected in pairs such that the two actions of each pair are similar in motion (have similar trajectories) and shape (have similar objects); however, the motion-shape relation is different. Source: HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences

The Privacy Annotated HMDB51 (PA-HMDB51) dataset is a video-based dataset for evaluating pirvacy protection in visual action recognition algorithms. The dataset contains both target task labels (action) and selected privacy attributes (skin color, face, gender, nudity, and relationship) annotated on a per-frame basis. Source: https://github.com/VITA-Group/PA-HMDB51

The TUM Kitchen dataset is an action recognition dataset that contains 20 video sequences captured by 4 cameras with overlapping views. The camera network captures the scene from four viewpoints with 25 fps, and every RGB frame is of the resolution 384×288 by pixels. The action labels are frame-wise, and provided for the left arm, the right arm and the torso separately. Source: Temporal Human Action Segmentation via Dynamic Clustering Image Source: https://ias.in.tum.de/dokuwiki/software/kitchen-activity-data

This dataset contains 70 (30 falls + 40 activities of daily living) sequences. Fall events are recorded with 2 Microsoft Kinect (RGB + Depth) cameras and corresponding accelerometric data. ADL events are recorded with only one camera and accelerometer. Sensor data was collected using PS Move (60Hz) and x-IMU (256Hz) devices.

ARID is a dataset for action recognition in dark videos. It consists of over 3,780 video clips with 11 action categories. Source: ARID: A New Dataset for Recognizing Action in the Dark

IndustReal is an ego-centric, multi-modal dataset where 27 participants are challenged to perform assembly and maintenance procedures on a construction-toy car. The dataset is annotated for action recognition, assembly state detection, and procedure step recognition. IndustReal includes 38 execution errors in a total of 84 videos, with 14 exclusive to validation and test sets and therefore suitable for testing the robustness of algorithms against unseen errors in procedural tasks. IndustReal offers open-source 3D models for all parts to promote the use of synthetic data for scalable approaches on this dataset, as well as reproducibility. All assembly parts used in the dataset are 3D printed. This ensures reproducibility and future availability of the model and allows for growth via community effort. IndustReal can be used to test methods towards (video) procedure understanding, industrial automation, and to quantify performance of augmented reality applications.

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

This dataset has the following citation: M. Soliman, M. Kamal, M. Nashed, Y. Mostafa, B. Chawky, D. Khattab, “ Violence Recognition from Videos using Deep Learning Techniques”, Proc. 9th International Conference on Intelligent Computing and Information Systems (ICICIS'19), Cairo, pp. 79-84, 2019. please use it in case of using the dataset in research or engineering purpose ) when we start our Graduation Project Violence Recognition from Videos we found that there is shortage in available datasets related to violence between individuals so we decide to create new big dataset with variety of scenes Content Our Dataset Contains 1000 Violence and 1000 non-violence videos collected from youtube videos, violence videos in our dataset contain many real street fights situations in several environments and conditions. also non-violence videos from our dataset are collected from many different human actions like sports, eating, walking …etc.

RoCoG-v2 (Robot Control Gestures) is a dataset intended to support the study of synthetic-to-real and ground-to-air video domain adaptation. It contains over 100K synthetically-generated videos of human avatars performing gestures from seven (7) classes. It also provides videos of real humans performing the same gestures from both ground and air perspectives

Simitate is a hybrid benchmarking suite targeting the evaluation of approaches for imitation learning. It consists on a dataset containing 1938 sequences where humans perform daily activities in a realistic environment. The dataset is strongly coupled with an integration into a simulator. RGB and depth streams with a resolution of 960×540 at 30Hz and accurate ground truth poses for the demonstrator's hand, as well as the object in 6 DOF at 120Hz are provided. Along with the dataset the 3D model of the used environment and labelled object images are also provided. Source: https://arxiv.org/abs/1905.06002 Image Source: https://github.com/raphaelmemmesheimer/simitate

UESTC RGB-D Varying-view action database contains 40 categories of aerobic exercise. We utilized 2 Kinect V2 cameras in 8 fixed directions and 1 round direction to capture these actions with the data modalities of RGB video, 3D skeleton sequences and depth map sequences.

This dataset is an OSN-transmitted (OSN = Online Social Network) version of the CASIA dataset. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (OSN = Online Social Network) version of the CASIA dataset. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (OSN = Online Social Network) version of the CASIA dataset. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (OSN = Online Social Network) version of the CASIA dataset. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the Columbia dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the Columbia dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the Columbia dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the Columbia dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the DSO dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the DSO dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the DSO dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the DSO dataset. Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

FLAG3D is a large-scale 3D fitness activity dataset with language instruction containing 180K sequences of 60 categories. FLAG3D features the following three aspects: 1) accurate and dense 3D human pose captured from advanced MoCap system to handle the complex activity and large movement, 2) detailed and professional language instruction to describe how to perform a specific activity, 3) versatile video resources from a high-tech MoCap system, rendering software, and cost-effective smartphones in natural environments. Source: FLAG3D: A 3D Fitness Activity Dataset with Language Instruction Image Source: https://arxiv.org/pdf/2212.04638v1.pdf

The FR-FS dataset contains 417 videos collected from FIV dataset and Pingchang 2018 Winter Olympic Games. FR-FS contains the critical movements of the athlete’s take-off, rotation, and landing. Among them, 276 are smooth landing videos, and 141 are fall videos. To test the generalization performance of our proposed model, we randomly select 50% of the videos from the fall and landing videos as the training set and the testing set.

This is the dataset for knowledge editing. It contains six tasks: ZsRE, $Wiki_{recent}$, $Wiki_{counterfact}$, WikiBio, ConvSent and Sanitation. This repo shows the former 4 tasks and you can get the data for ConvSent and Sanitation from their original papers.

The MA-52 dataset provides the whole-body perspective including gestures, upper- and lower-limb movements, attempting to reveal comprehensive micro-action cues. In detail, MA-52 contains 52 micro-action categories along with seven body part labels, and encompasses a full array of realistic and natural micro-actions, accounting for 205 participants and 22,422 video instances collated from the psychological interviews

This dataset is an OSN-transmitted (Online Social Network) version of the NIST dataset (https://www.nist.gov/itl/iad/mig/nimble-challenge-2017-evaluation). Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.

This dataset is an OSN-transmitted (Online Social Network) version of the NIST dataset (https://www.nist.gov/itl/iad/mig/nimble-challenge-2017-evaluation). Unfortunately, OSNs automatically apply operations like compression and resizing, which reduce valuable information necessary for image forgery detection. As a result, this dataset presents a greater challenge for forgery detection compared to the original non-OSN-transmitted version. The dataset is available here: https://github.com/HighwayWu/ImageForensicsOSN - more specifically: https://drive.google.com/file/d/1uMNZdhX3bYAZNcVGlkCvrnj5lSLW1ld5/view?usp=sharing and was presented in: [Wu et al., 2022] Wu, H., Zhou, J., Tian, J., Liu, J., and Qiao, Y. (2022). Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security.