태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
Human Activity Recognition is the problem of identifying events performed by humans given a video input. It is formulated as a binary (or multiclass) classification problem of outputting activity class labels. Activity Recognition is an important problem with many societal applications including smart surveillance, video search/retrieval, intelligent robots, and other monitoring systems.
Source: Learning Latent Sub-events in Activity Videos Using Temporal Attention Filters
출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.
First-Person Hand Action Benchmark is a collection of RGB-D video sequences comprised of more than 100K frames of 45 daily hand action categories, involving 26 different objects in several hand configurations. Source: First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations
We present a comprehensive framework for egocentric interaction recognition using markerless 3D annotations of two hands manipulating objects. To this end, we propose a method to create a unified dataset for egocentric 3D interaction recognition. Our method produces annotations of the 3D pose of two hands and the 6D pose of the manipulated objects, along with their interaction labels for each frame. Our dataset, called H2O (2 Hands and Objects), provides synchronized multi-view RGB-D images, interaction labels, object classes, ground-truth 3D poses for left & right hands, 6D object poses, ground-truth camera poses, object meshes and scene point clouds. To the best of our knowledge, this is the first benchmark that enables the study of first-person actions with the use of the pose of both left and right hands manipulating objects and presents an unprecedented level of detail for egocentric 3D interaction recognition. We further propose the method to predict interaction classes by estimating the 3D pose of two hands and the 6D pose of the manipulated objects, jointly from RGB images. Our method models both inter- and intra-dependencies between both hands and objects by learning the topology of a graph convolutional network that predicts interactions. We show that our method facilitated by this dataset establishes a strong baseline for joint hand-object pose estimation and achieves state-of-the-art accuracy for first person interaction recognition.
HAA500 is a manually annotated human-centric atomic action dataset for action recognition on 500 classes with over 591k labeled frames. Unlike existing atomic action datasets, where coarse-grained atomic actions were labeled with action-verbs, e.g., "Throw", HAA500 contains fine-grained atomic actions where only consistent actions fall under the same label, e.g., "Baseball Pitching" vs "Free Throw in Basketball", to minimize ambiguities in action classification. HAA500 has been carefully curated to capture the movement of human figures with less spatio-temporal label noises to greatly enhance the training of deep neural networks. Source: HAA500: Human-Centric Atomic Action Dataset with Curated Videos
The Watch-n-Patch dataset was created with the focus on modeling human activities, comprising multiple actions in a completely unsupervised setting. It is collected with Microsoft Kinect One sensor for a total length of about 230 minutes, divided in 458 videos. 7 subjects perform human daily activities in 8 offices and 5 kitchens with complex backgrounds. Moreover, skeleton data are provided as ground truth annotations. Source: Head Detection with Depth Images in the Wild Image Source: https://openaccess.thecvf.com/content_cvpr_2015/papers/Wu_Watch-n-Patch_Unsupervised_Understanding_2015_CVPR_paper.pdf
A new and challenging video database of dynamic scenes that more than doubles the size of those previously available. This dataset is explicitly split into two subsets of equal size that contain videos with and without camera motion to allow for systematic study of how this variable interacts with the defining dynamics of the scene per se. Source: Temporal Residual Networks for Dynamic Scene Recognition
EgoExoLearn is a fascinating dataset designed to bridge the gap between egocentric and exocentric views of procedural activities. 1. What Is EgoExoLearn? EgoExoLearn is a large-scale dataset that emulates how humans learn by observing others. It focuses on the process of asynchronous demonstration following. Participants in the dataset record egocentric videos as they perform tasks. These videos are guided by exocentric-view demonstration videos. In simpler terms, imagine someone watching a demonstration video (from an external perspective) and then replicating the same task while recording their own point-of-view video. 2. Dataset Details: EgoExoLearn dataset spans 120 hours and covers scenarios from daily life and specialized laboratories. It contains: - Egocentric videos: These are recorded by individuals executing tasks. - Demonstration videos: These show the same tasks from an external viewpoint. - Gaze data: High-quality gaze information accompanies the videos. - Multimodal annotations: Detailed annotations provide context and insights. 3. Applications and Benchmarks: The EgoExoLearn dataset serves as a playground for modeling the human ability and thus provides a playground to bridge asynchronous procedural actions from different viewpoints. It enables new benchmarks such as: - Cross-view association: Linking actions observed from different perspectives. - Cross-view action planning: Anticipating and planning actions based on both ego and exo views. - Cross-view referenced skill assessment: Evaluating skills across viewpoints. Using Exo-view demonstrations as guidance for better ego-view skill assessment. 4. Why Is It Important? - Understanding how we map others' activities into our own point of view is a fundamental human skill. - EgoExoLearn paves the way for creating AI agents capable of seamlessly learning by observing humans in the real world. (1) EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric .... https://arxiv.org/html/2403.16182v1. (2) EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric .... https://egoexolearn.github.io/. (3) EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric .... https://arxiv.org/abs/2403.16182. (4) EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric .... https://allainews.com/item/egoexolearn-a-dataset-for-bridging-asynchronous-ego-and-exo-centric-view-of-procedural-activities-in-real-world-2024-03-26/. (5) undefined. https://github.com/OpenGVLab/EgoExoLearn/.
EPIC-SOUNDS is a large scale dataset of audio annotations capturing temporal extents and class labels within the audio stream of the egocentric videos from EPIC-KITCHENS-100. EPIC-SOUNDS includes 78.4k categorised and 39.2k non-categorised segments of audible events and actions, distributed across 44 classes. Source: Epic-Sounds: A Large-scale Dataset of Actions That Sound
Contains 60 female and 30 male actors performing a collection of 20 predefined everyday actions and sports movements, and one self-chosen movement. Source: MoVi: A Large Multipurpose Motion and Video Dataset
RareAct is a video dataset of unusual actions, including actions like “blend phone”, “cut keyboard” and “microwave shoes”. It aims at evaluating the zero-shot and few-shot compositionality of action recognition models for unlikely compositions of common action verbs and object nouns. It contains 122 different actions which were obtained by combining verbs and nouns rarely co-occurring together in the large-scale textual corpus from HowTo100M, but that frequently appear separately. Source: https://github.com/antoine77340/RareAct Image Source: https://github.com/antoine77340/RareAct
Casia V1 is a dataset for forgery classification. Casia V1+ is a modification of the Casia V1 dataset proposed by Chen et al. that replaces authentic images that also exist in Casiav2 with images from the COREL dataset to avoid data contamination.
Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset
The AI City Challenge, hosted at CVPR 2024, focuses on harnessing AI to enhance operational efficiency in physical settings such as retail and warehouse environments, and Intelligent Traffic Systems (ITS). It aims to utilize AI for actionable insights from sensor data, like camera feeds, to improve traffic safety and transportation outcomes. This year, the challenge spotlights two key areas with significant potential: retail business and ITS.
TAPOS is a new dataset developed on sport videos with manual annotations of sub-actions, and conduct a study on temporal action parsing on top. A sport activity usually consists of multiple sub-actions and that the awareness of such temporal structures is beneficial to action recognition. TAPOS contains 16,294 valid instances in total, across 21 action classes. These instances have a duration of 9.4 seconds on average. The number of instances within each class is different, where the largest class high jump has over 1,600 instances, and the smallest class beam has 200 instances. The average number of sub-actions also varies from class to class, where parallel bars has 9 sub-actions on average, and long jump has 3 sub-actions on average. All instances are split into train, validation and test sets, of sizes 13094, 1790, and 1763, respectively.
The Collective Activity Dataset contains 5 different collective activities: crossing, walking, waiting, talking, and queueing and 44 short video sequences some of which were recorded by consumer hand-held digital camera with varying view point. Source: http://vhosts.eecs.umich.edu/vision//activity-dataset.html Image Source: http://vhosts.eecs.umich.edu/vision//activity-dataset.html
COVERAGE contains copymove forged (CMFD) images and their originals with similar but genuine objects (SGOs). COVERAGE is designed to highlight and address tamper detection ambiguity of popular methods, caused by self-similarity within natural images. In COVERAGE, forged–original pairs are annotated with (i) the duplicated and forged region masks, and (ii) the tampering factor/similarity metric. For benchmarking, forgery quality is evaluated using (i) computer vision-based methods, and (ii) human detection performance. Source: COVERAGE
EgoHOS is a labeled dataset consisting of 11243 egocentric images with per-pixel segmentation labels of hands and objects being interacted with during a diverse array of daily activities. The data are collected form multiple sources: 7,458 frames from Ego4D, 2,212 frames from EPIC-KITCHEN, 806 frames from THU-READ, and 350 frames of our own collected egocentric videos with people playing Escape Room. This dataset is designed for tasks including hand state classification, video activity recognition, 3D mesh reconstruction of hand-object interactions, and video inpainting of hand-object foregrounds in egocentric videos. Source: Fine-grained Egocentric Hand-object Segmentation: Dataset, Model, and Applications Image Source: https://arxiv.org/pdf/2208.03826v1.pdf
Home Action Genome is a large-scale multi-view video database of indoor daily activities. Every activity is captured by synchronized multi-view cameras, including an egocentric view. There are 30 hours of vides with 70 classes of daily activities and 453 classes of atomic actions.
We describe the 2020 edition of the DeepMind Kinetics human action dataset, which replenishes and extends the Kinetics-700 dataset. In this new version, there are at least 700 video clips from different YouTube videos for each of the 700 classes. This paper details the changes introduced for this new release of the dataset and includes a comprehensive set of statistics as well as baseline results using the I3D network.
PSI-AVA is a dataset designed for holistic surgical scene understanding. It contains approximately 20.45 hours of the surgical procedure performed by three expert surgeons and annotations for both long-term (Phase and Step recognition) and short-term reasoning (Instrument detection and novel Atomic Action recognition) in robot-assisted radical prostatectomy videos. Source: Towards Holistic Surgical Scene Understanding Image Source: https://arxiv.org/pdf/2212.04582v2.pdf
Kinetics-100 is a dataset split created from the Kinetics dataset to evaluate the performance of few-shot action recognition models. 100 classes are randomly selected from a total of 400 categories, each composed of 100 examples. The 100 classes are further split into 64, 12, and 24 non-overlapping classes to use as the meta-training set, meta-validation set, and meta-testing set, respectively. Link to the selected samples can be found here: https://github.com/ffmpbgrnn/CMN/tree/master/kinetics-100
The MISAW data set is composed of 27 sequences of micro-surgical anastomosis on artificial blood vessels performed by 3 surgeons and 3 engineering students. The dataset contained video, kinematic, and procedural descriptions synchronized at 30Hz. The procedural descriptions contained phases, steps, and activities performed by the participants. Source: MIcro-Surgical Anastomose Workflow recognition challenge report
OPERAnet is a multimodal activity recognition dataset acquired from radio frequency and vision-based sensors. Approximately 8 hours of annotated measurements are provided, which are collected across two different rooms from 6 participants performing 6 activities, namely, sitting down on a chair, standing from sit, lying down on the ground, standing from the floor, walking and body rotating. The dataset has been acquired from four synchronized modalities for the purpose of passive Human Activity Recognition (HAR) as well as localization and crowd counting.
Includes egocentric videos containing hands in the wild. Source: Analysis of Hand Segmentation in the Wild
This is a 3D action recognition dataset, also known as 3D Action Pairs dataset. The actions in this dataset are selected in pairs such that the two actions of each pair are similar in motion (have similar trajectories) and shape (have similar objects); however, the motion-shape relation is different. Source: HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences
NTU RGB+D 2D is a curated version of NTU RGB+D often used for skeleton-based action prediction and synthesis. It contains less number of actions.
The Privacy Annotated HMDB51 (PA-HMDB51) dataset is a video-based dataset for evaluating pirvacy protection in visual action recognition algorithms. The dataset contains both target task labels (action) and selected privacy attributes (skin color, face, gender, nudity, and relationship) annotated on a per-frame basis. Source: https://github.com/VITA-Group/PA-HMDB51
The TUM Kitchen dataset is an action recognition dataset that contains 20 video sequences captured by 4 cameras with overlapping views. The camera network captures the scene from four viewpoints with 25 fps, and every RGB frame is of the resolution 384×288 by pixels. The action labels are frame-wise, and provided for the left arm, the right arm and the torso separately. Source: Temporal Human Action Segmentation via Dynamic Clustering Image Source: https://ias.in.tum.de/dokuwiki/software/kitchen-activity-data
This dataset contains 70 (30 falls + 40 activities of daily living) sequences. Fall events are recorded with 2 Microsoft Kinect (RGB + Depth) cameras and corresponding accelerometric data. ADL events are recorded with only one camera and accelerometer. Sensor data was collected using PS Move (60Hz) and x-IMU (256Hz) devices.
ARID is a dataset for action recognition in dark videos. It consists of over 3,780 video clips with 11 action categories. Source: ARID: A New Dataset for Recognizing Action in the Dark
IndustReal is an ego-centric, multi-modal dataset where 27 participants are challenged to perform assembly and maintenance procedures on a construction-toy car. The dataset is annotated for action recognition, assembly state detection, and procedure step recognition. IndustReal includes 38 execution errors in a total of 84 videos, with 14 exclusive to validation and test sets and therefore suitable for testing the robustness of algorithms against unseen errors in procedural tasks. IndustReal offers open-source 3D models for all parts to promote the use of synthetic data for scalable approaches on this dataset, as well as reproducibility. All assembly parts used in the dataset are 3D printed. This ensures reproducibility and future availability of the model and allows for growth via community effort. IndustReal can be used to test methods towards (video) procedure understanding, industrial automation, and to quantify performance of augmented reality applications.
A dataset which provides detailed annotations for activity recognition. Source: Recognizing Fine-Grained and Composite Activities using Hand-Centric Features and Script Data
Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset
This dataset has the following citation: M. Soliman, M. Kamal, M. Nashed, Y. Mostafa, B. Chawky, D. Khattab, “ Violence Recognition from Videos using Deep Learning Techniques”, Proc. 9th International Conference on Intelligent Computing and Information Systems (ICICIS'19), Cairo, pp. 79-84, 2019. please use it in case of using the dataset in research or engineering purpose ) when we start our Graduation Project Violence Recognition from Videos we found that there is shortage in available datasets related to violence between individuals so we decide to create new big dataset with variety of scenes Content Our Dataset Contains 1000 Violence and 1000 non-violence videos collected from youtube videos, violence videos in our dataset contain many real street fights situations in several environments and conditions. also non-violence videos from our dataset are collected from many different human actions like sports, eating, walking …etc.
RoCoG-v2 (Robot Control Gestures) is a dataset intended to support the study of synthetic-to-real and ground-to-air video domain adaptation. It contains over 100K synthetically-generated videos of human avatars performing gestures from seven (7) classes. It also provides videos of real humans performing the same gestures from both ground and air perspectives
Simitate is a hybrid benchmarking suite targeting the evaluation of approaches for imitation learning. It consists on a dataset containing 1938 sequences where humans perform daily activities in a realistic environment. The dataset is strongly coupled with an integration into a simulator. RGB and depth streams with a resolution of 960×540 at 30Hz and accurate ground truth poses for the demonstrator's hand, as well as the object in 6 DOF at 120Hz are provided. Along with the dataset the 3D model of the used environment and labelled object images are also provided. Source: https://arxiv.org/abs/1905.06002 Image Source: https://github.com/raphaelmemmesheimer/simitate
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.