Papers with Code 아카이브

Video

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
34
평가표
0
보존된 평가 기록
0

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

MoB

A dataset of cartoon video clips. For each video clip, the presence or absence of each feature was marked by the annotators. Malicious: Video content which may not be suitable for viewing by toddlers and pre-schoolers includes a set of clearly defined, trivial and intu-itive features as well as some complex and subtle audio andvideo features. The former includes forms of obscenity and violence on a higher level e.g. nudity, gore, etc. while the latter includes elements such as fast repetitive motion, loud music, disgusting and scary characters, smashing people or things, forms of aggression, loud music, screaming or shout-ing, gunshots and explosions. Benign: Educational videos and videos of nursery rhymes are usually considered to be appropriate for the toddlers and pre-schoolers; in fact some experts recommend letting kids watch them for a limited number of hours.Benign videos are characterized by a slower tempo, softer music or sound effects, moderate-paced motions, and soft-toned conversations. Most importantly, benign video content should not contain any indicators of malicious content as discussed earlier.

MultiviewC

The MultiviewC dataset mainly contributes to multiview cattle action recognition, 3D objection detection and tracking. We build a novel synthetic dataset MultiviewC through UE4 based on real cattle video dataset which is offered by CISRO. The format of our data set has been adjusted on the basis of MultiviewX for set-up, annotation and files structure.

NTU-X

NTU-X is an extended version of popular NTU dataset.

ODMS

ODMS is a dataset for learning Object Depth via Motion and Segmentation. ODMS training data are configurable and extensible, with each training example consisting of a series of object segmentation masks, camera movement distances, and ground truth object depth. As a benchmark evaluation, the dataset provides four ODMS validation and test sets with 15,650 examples in multiple domains, including robotics and driving. Source: https://github.com/griffbr/ODMS Image Source: https://github.com/griffbr/ODMS

PersonPath22

PersonPath22 is a large-scale multi-person tracking dataset containing 236 videos captured mostly from static-mounted cameras, collected from sources where we were given the rights to redistribute the content and participants have given explicit consent. Each video has ground-truth annotations including both bounding boxes and tracklet-ids for all the persons in each frame.

PUMaVOS

PUMaVOS is a dataset of challenging and practical use cases inspired by the movie production industry. Partial and Unusual Masks for Video Object Segmentation (PUMaVOS) dataset has the following properties: - 24 videos, 21187 densely-annotated frames; - Covers complex practical use cases such as object parts, frequent occlusions, fast motion, deformable objects and more; - Average length of the video is 883 frames or 29s, with the longer ones spanning 1min; - Fully densely annotated at 30FPS; - Benchmark-oriented: no separation into training/test, designed to be as diverse as possible to test your models; - 100% open and free to download.

RAD

The dataset is useful for query-adaptive video summarization and annotated with diversity and query-specific relevance labels. Source: Query-adaptive Video Summarization via Quality-aware Relevance Estimation

RGB-DAVIS Dataset

Used to show systematic performance improvement in applications such as high frame-rate video synthesis, feature/corner detection and tracking, as well as high dynamic range image reconstruction. Source: Joint Filtering of Intensity Images and Neuromorphic Events for High-Resolution Noise-Robust Imaging

RISE

RISE is a large-scale video dataset for Recognizing Industrial Smoke Emissions. A citizen science approach was adopted to collaborate with local community members to annotate whether a video clip has smoke emissions. The dataset contains 12,567 clips from 19 distinct views from cameras that monitored three industrial facilities. These daytime clips span 30 days over two years, including all four seasons. Source: https://arxiv.org/abs/2005.06111 Image Source: https://github.com/CMU-CREATE-Lab/deep-smoke-machine

SC_burst

Contains16 burst images using smartphones for burst/video denoising, restoration, and enhancement tasks. The raw format are unified and saved SC_burst in ".MAT", where the raw data and metadata are stored.

SEPE 8K

SEPE 8K dataset is made of 40 different 8K (8192 x 4320) video sequences and 40 variant 8K (8192 x 5464) images. The video sequences were captured at a framerate of 29.97 frames per second (FPS) and had been encoded into videos using AVC/H.264, HEVC/H.265, and AV1 codecs at resolutions from 8K to 480p. The images, video sequences, encoded videos, and various other statistics related to the media that make the dataset are stored online, published, and maintained on the repo on GitHub for non-commercial use. this proposed dataset is - as far as we know - the first to publish true 8K natural sequences; thus, it is important for the next level of applications dealing with multimedia such as video quality assessment, super-resolution, video coding, video compression, and many more. Paper: https://doi.org/10.1145/3587819.3592560

Skeletics 152

A curated and 3-D pose-annotated subset of RGB videos sourced from Kinetics-700, a large-scale action dataset. Source: Quo Vadis, Skeleton Action Recognition ?

Skeleton-Mimetics

A dataset derived from the recently introduced Mimetics dataset. Source: Quo Vadis, Skeleton Action Recognition ?

Stanford-ECM

Stanford-ECM is an egocentric multimodal dataset which comprises about 27 hours of egocentric video augmented with heart rate and acceleration data. The lengths of the individual videos cover a diverse range from 3 minutes to about 51 minutes in length. A mobile phone was used to collect egocentric video at 720x1280 resolution and 30 fps, as well as triaxial acceleration at 30Hz. The mobile phone was equipped with a wide-angle lens, so that the horizontal field of view was enlarged from 45 degrees to about 64 degrees. A wrist-worn heart rate sensor was used to capture the heart rate every 5 seconds. The phone and heart rate monitor was time-synchronized through Bluetooth, and all data was stored in the phone’s storage. Piecewise cubic polynomial interpolation was used to fill in any gaps in heart rate data. Finally, data was aligned to the millisecond level at 30 Hz. Source: http://ai.stanford.edu/~syyeung/ecm_dataset/egocentric_multimodal.html Image Source: http://ai.stanford.edu/~syyeung/ecm_dataset/egocentric_multimodal.html

SVBench

Dataset Card for SVBench This dataset card aims to provide a comprehensive overview of the SVBench dataset, including its purpose, structure, and sources. For details, see our Project, Paper and GitHub repository. ## Dataset Details ### Dataset Description SVBench is the first benchmark specifically designed to evaluate long-context streaming video understanding through temporal multi-turn question-answering (QA) chains. It addresses the limitations of existing video understanding benchmarks by emphasizing continuous temporal reasoning over streaming video data. The dataset includes: - 1,353 streaming videos (average duration: 1–8 minutes) - 49,979 QA pairs organized into multi-turn dialogues - Temporal linkages between QA chains to simulate real-world streaming scenarios Languages - NLP: English & Chinese (bilingual annotations) License - Apache-2.0 Dataset Sources - YT-Temporal-1B|YouCook2|MovieChat|Panda-70M|Ego4D|ActivityNet. ## Uses Download SVBench dataset from Hugging Face: git clone https://huggingface.co/yzy666/SVBench Intended Use - Evaluate long-context reasoning capabilities of Large Vision-Language Models (LVLMs) in streaming video scenarios. - Benchmark temporal understanding through multi-turn dialogues. - Support research in streaming video QA, activity forecasting, and interactive AI assistants. Direct Use - Training or fine-tuning LVLMs for streaming video tasks. - Testing model robustness in handling dynamic, time-sensitive queries. - Comparative analysis of open-source vs. closed-source LVLMs. Restrictions - Videos and annotations are for research purposes only. - Users must comply with the original licenses of datasets. ## Dataset Structure - Folder Structure Tree: SVBench/ ├── Con/ │ ├── Con_EN/ │ └── Con_ZH/ ├── Dialogue/ │ ├── Dialogue_EN/ │ └── Dialogue_ZH/ ├── Meta/ │ ├── Meta_EN/ │ └── Meta_ZH/ ├── Path/ ├── Src/ ├── Streaming/ │ ├── Streaming_EN/ │ └── Streaming_ZH/ ├── Video/ └── Your_Model_Name/ ├── dialogue/ │ └── --NDulaHyrE.json └── streaming/ └── -4h8cuweoKo.json - Dataset Division: - - Training Set: 42,605 QA pairs and 1,153 videos - Testing Set: 7,374 QA pairs and 200 videos ## Data Fields | Key | Description | | ------------------------- | ------------------------------------------------------------ | | Video_Name | Unique identifier or title of the video file. | | Sort_of_Set | Category or type of the dataset subset (e.g., "Train", "Test"). | | Path_of_QandA | File path to the question-answer pairs. | | Path_of_Con | Path to relationship files. | | Path_of_StreamingPathData | Path to Q&A sequence for streaming evaluation. Each streaming path contains all Q&A sequences in streaming order within the path. | | Path_of_Dialogue | Path to Q&A sequence for dialogue evaluation. Each dialogue contains all Q&A sequences in order within the dialogue. | | Path_of_Streaming | Path to Q&A sequence for streaming evaluation represented only by serial numbers (e.g., the path [[0,0],[1,2],[2,3]...] indicates starting from the 1st question of the 1st chain, then proceeding to the 3rd question of the 2nd chain, and then to the 4th question of the 3rd chain, and so on). | | Path_of_Video | Absolute file path to the raw video file. | | Video_Duration | Total duration of the video in seconds. | | Source_of_Dataset | Origin of the dataset. | ## Leaderboard Submission Submit results via https://forms.gle/Rmi6u4WGhyEZ2X7g8. Submission Instructions: 1. Save result files following the case structure under the SVBench/[Your_Model_Name] directory 2. Compress the entire [Your_Model_Name] folder into a ZIP archive 3. Upload the generated ZIP file through this submission portal Important Notes: - Maintain the exact directory hierarchy: [Your_Model_Name]/[dialogue/streaming]/[Video_Name.json] - Ensure your model name follows exact capitalization (case-sensitive) - Package only the required result files (avoid including source code/executables) Submission Form img ## Dataset Annotation See GitHub repository for details. Semi-automated annotation using a hybrid approach: - Automatic QA Generation: Leveraging GPT-4 to generate dialogue chains based on video transcripts. - Human Verification: Annotators validate temporal links and refine QA pairs for consistency. - Temporal Linking: Artificial alignment of QA chains across video segments. - Quality Control: Inter-annotator agreement >90% for temporal links. ## Citation If you find our data useful, please consider citing our work! @article{yang2025svbench, title={SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding}, author={Yang, Zhenyu and Hu, Yuhang and Du, Zemin and Xue, Dizhan and Qian, Shengsheng and Wu, Jiahong and Yang, Fan and Dong, Weiming and Xu, Changsheng}, journal={arXiv preprint arXiv:2502.10810}, year={2025} }

VFITex

To test interpolation performance on various texture types, we developed a new test set, VFITex, which contains twenty 100-frame UHD or HD videos at 24, 30 or 50 FPS, collected from the Xiph, Mitch Martinez Free 4K Stock Footage, UVG database and pexels.com. This dataset covers diverse textured scenes, including crowds, flags, foliage, animals, water, leaves, fire and smoke. HD patches were center-cropped from the UHD sequences, preserving the original UHD characteristics. All frames in each sequence were used for evaluation, totaling 940 quintuplets.

VFP290K

Vision-based Fallen Person (VFP290K) is a novel, large-scale dataset for the detection of fallen persons composed of fallen person images collected in various real-world scenarios. VFP290K consists of 294,714 frames of fallen persons extracted from 178 videos, including 131 scenes in 49 locations.

VTC

VTC is a large-scale multimodal dataset containing video-caption pairs (~300k) alongside comments that can be used for multimodal representation learning.

WiFall

WiFall dataset contains data related to fall detection, action recognition and people id identification in a meeting room scenario. The dataset provides synchronised CSI, RSSI, and timestamp for each sample.

[[Talk!!Person]]How do I talk to a person at Expedia?

You can contact Expedia by phone at +1-805-330-4056 if you're in Mexico, or +1-888-829-0881 if you're in the United States. Both numbers are available 24 hours a day and offer support in Spanish to help you with any travel-related questions. How to speak to an Expedia agent? To speak to an Expedia agent, dial +1-805-330-4056 from Mexico or +1-888-829-0881 from the U.S. A Spanish-speaking advisor will answer your call and assist you with reservations, changes, cancellations, refunds, or other requests. How to speak to an Expedia agent? In addition to phone support, Expedia offers live chat support on its official website (Expedia.com). You can choose your preferred channel and receive personalized help in Spanish, 24 hours a day. How to request a refund on Expedia? If you wish to request a refund, please call +1-888-829-0881 (United States) or +1-805-330-4056 (Mexico). An agent will review your reservation, verify its eligibility for a refund, and guide you through every step of the process. How do I speak to an Expedia advisor? You can speak to an advisor by calling +1-888-829-0881 or +1-805-330-4056. Both offer support in Spanish 24/7. You can also use the online chat for immediate assistance from the official website.

AirLetters

This large collection of over 161,000 video-label pairs of video clips, shows humans drawing letters and digits in the air, and is used to evaluate a model’s ability to classify articulated motions correctly. Unlike existing video datasets, AirLetters’ accurate classification predictions rely on discerning motion patterns and integrating information presented by the video over time (i.e., over many frames of video). That study revealed that while trivial for humans, accurate representations of complex articulated motions remain an open problem for end-to-end learning for video understanding models.

ANUBIS

ANUBIS is a large-scale human skeleton dataset containing 80 actions. Compared with previously collected datasets, ANUBIS is advantageous in the following four aspects: (1) employing more recently released sensors; (2) containing novel back view; (3) encouraging high enthusiasm of subjects; (4) including actions of the COVID pandemic era.

Bukva

We introduce a video dataset Bukva for Russian Dactyl Recognition task. Bukva dataset size is about 27 GB, and it contains 3757 RGB videos with more than 101 samples for each RSL alphabet sign, including dynamic ones. The dataset is divided into training set and test set by subject user_id. The training set includes 3097 videos, and the test set includes 660 videos. The total video recording time is ~4 hours. About 17% of the videos are recorded in HD format, and 70% of the videos are in FullHD resolution.

CAShift

CAShift is the first multiple normality shift-aware Log-Based Anomaly Detection (LAD) dataset specifically designed for cloud systems, which considers different software roles in cloud systems and attack behavior among cloud components.

CausalChaos!

CausalChaos! is a dataset for causal video question answering. It is based on Tom and Jerry cartoons. It features longer causal chains embedded in dynamic visual scenes. It also features challenging incorrect options, especially, Causal Confusion set which contains causally confounding incorrect options. All these factors prove to be challenging for current VLMs and other traditional Video Question Answering models.

ChronoMagic-ProH

Description - Venue: NeurIPS 2024 D&B Spotlight - Repository: Code, Page, Data - Paper: arxiv.org/abs/2406.18522 - Point of Contact: Shenghai Yuan ## Citation If you find our paper and code useful in your research, please consider giving a star and citation. BibTeX @article{yuan2024chronomagic, title={Chronomagic-bench: A benchmark for metamorphic evaluation of text-to-time-lapse video generation}, author={Yuan, Shenghai and Huang, Jinfa and Xu, Yongqi and Liu, Yaoyang and Zhang, Shaofeng and Shi, Yujun and Zhu, Ruijie and Cheng, Xinhua and Luo, Jiebo and Yuan, Li}, journal={arXiv preprint arXiv:2406.18522}, year={2024} }

CinePile: A Long Video Question Answering Dataset and Benchmark

CinePile is a question-answering-based, long-form video understanding dataset. It has been created using advanced large language models (LLMs) with human-in-the-loop pipeline leveraging existing human-generated raw data. It consists of approximately 300,000 training data points and 5,000 test data points.

CloudCast

A satellite-based dataset called "CloudCast". It consists of 70080 images with 10 different cloud types for multiple layers of the atmosphere annotated on a pixel level. The spatial resolution of the dataset is 928 × 1530 pixels (3 × 3 km per pixel) with 15-min intervals between frames for the period January 1, 2017, to December 31, 2018. All frames are centered and projected over Europe. Source: CloudCast: A Satellite-Based Dataset and Baseline for Forecasting Clouds

ConsisID-preview-Data

Description - Repository: Code, Page, Data - Paper: arxiv.org/abs/2411.17440 - Point of Contact: Shenghai Yuan ## Citation If you find our paper and code useful in your research, please consider giving a star and citation. BibTeX @article{yuan2024identity, title={Identity-Preserving Text-to-Video Generation by Frequency Decomposition}, author={Yuan, Shenghai and Huang, Jinfa and He, Xianyi and Ge, Yunyuan and Shi, Yujun and Chen, Liuhan and Luo, Jiebo and Yuan, Li}, journal={arXiv preprint arXiv:2411.17440}, year={2024} }

Custom Spatio-Temporal Action Video Dataset

This spatio-temporal actions dataset for video understanding consists of 4 parts: original videos, cropped videos, video frames, and annotation files. This dataset uses a proposed new multi-person annotation method of spatio-temporal actions. First, we use ffmpeg to crop the videos and frame the videos; then use yolov5 to detect human in the video frame, and then use deep sort to detect the ID of the human in the video frame. By processing the detection results of yolov5 and deep sort, we can get the annotation file of the spatio-temporal action dataset to complete the work of customizing the spatio-temporal action dataset.

DARai

Daily Activity Recordings for Artificial Intelligence (DARai, pronounced "Dahr-ree") is a multimodal, hierarchically annotated dataset constructed to understand human activities in real-world settings. DARai consists of continuous scripted and unscripted recordings of 50 participants in 10 different environments, totaling over 200 hours of data from 20 sensors including multiple camera views, depth and radar sensors, wearable inertial measurement units (IMUs), electromyography (EMG), insole pressure sensors, biomonitor sensors, and gaze tracker. To capture the complexity in human activities, DARai is annotated at three levels of hierarchy: (i) high-level activities (L1) that are independent tasks, (ii) lower-level actions (L2) that are patterns shared between activities, and (iii) fine-grained procedures (L3) that detail the exact execution steps for actions. The dataset annotations and recordings are designed so that 22.7% of L2 actions are shared between L1 activities and 14.2% of L3 procedures are shared between L2 actions. The overlap and unscripted nature of DARai allows counterfactual activities in the dataset. Experiments with various machine learning models showcase the value of DARai in uncovering important challenges in human-centered applications. Specifically, we conduct unimodal and multimodal sensor fusion experiments for recognition, temporal localization, and future action anticipation across all hierarchical annotation levels. To highlight the limitations of individual sensors, we also conduct domain-variant experiments that are enabled by DARai's multi-sensor and counterfactual activity design setup. The code, documentation, and dataset are available at the dedicated DARai website.

DeVAn

DeVAn is a multi-modal dataset containing 8.5K video clips carefully selected from previously published YouTube-based video datasets (YouTube-8M and YT-Temporal-1B) that integrate visual and auditory information. Over the span of 10 months, a team of 24 human annotators (college and graduate level students) created 5 short captions (1 sentence each) and 5 long summaries (3-10 sentences) for each video clip, resulting in a rich and comprehensive human-annotated dataset that serves as a robust ground truth for subsequent model training and evaluation.

FollowMe Vehicle Behaviour Prediction Dataset

This dataset is a result of a study that was created to assess drivers behaviors when following a lead vehicle. The driving simulator study used a simulated suburban environment for collecting driver behavior data while following a lead vehicle driving through various unsignalized intersections. The driving environment had two lanes in each direction and a dedicated left-turn lane for the intersection. The experiment was deployed on a miniSim Driving Simulator. We programmed the lead vehicle ran- domly turn left, right or go straight through the intersections. In total we had 2(traffic density) × 2(speed level) × 3 = 12 scenarios for each participant to be tested on. We split the data into train, validation and test sets. The setup for the task is to observe 1 second of trajectories and predict the next 3,5 and 8 seconds.

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.