Videezy4K
To evaluate the performance on 4K burst images/video, we collect several clips from website. The dataset can be download from : https://drive.google.com/file/d/1YDljUONvyKUO24smTx__CUH_4Zxhle09/view?usp=sharing
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.
To evaluate the performance on 4K burst images/video, we collect several clips from website. The dataset can be download from : https://drive.google.com/file/d/1YDljUONvyKUO24smTx__CUH_4Zxhle09/view?usp=sharing
The dataset contains 10 reference videos and 1467 degraded videos. The videos were transmitted via Microsoft Teams calls in 83 different network conditions and contain various typical videoconferencing impairments. It also includes P.910 Crowd subjective video MOS ratings (see paper for more info).
VILT is a new benchmark collection of tasks and multimodal video content. The video linking collection includes annotations from 10 (recipe) tasks, which the annotators chose from a random subset of the collection of 2,275 high-quality 'Wholefoods' recipes. There are linking annotations for 61 query steps across these tasks which contain cooking techniques, chosen from the 189 total recipe steps. As each method results in approximately 10 videos to annotate, the collection consists of 831 linking judgments. Source: VILT: Video Instructions Linking for Complex Tasks Image source: https://arxiv.org/pdf/2208.10858v1.pdf
VISEM-Tracking is a dataset consisting of 20 video recordings of 30s of spermatozoa with manually annotated bounding-box coordinates and a set of sperm characteristics analyzed by experts in the domain. It is an extension of the previously published VISEM dataset. In addition to the annotated data, unlabeled video clips are provided for easy-to-use access and analysis of the data. Source: VISEM-Tracking: Human Spermatozoa Tracking Dataset Image Source: https://arxiv.org/pdf/2212.02842v1.pdf
VISOR is a dataset of pixel annotations and a benchmark suite for segmenting hands and active objects in egocentric video. VISOR annotates videos from EPIC-KITCHENS, and it contains 272K manual semantic masks of 257 object classes, 9.9M interpolated dense masks, and 67K hand-object relations, covering 36 hours of 179 untrimmed videos. Source: EPIC-KITCHENS VISOR Benchmark Video Segmentations and Object Relations Image Source: https://arxiv.org/pdf/2209.13064v1.pdf
We consider the task of temporal human action localization in lifestyle vlogs. We introduce a novel dataset consisting of manual annotations of temporal localization for 13,000 narrated actions in 1,200 video clips. We present an extensive analysis of this data, which allows us to better understand how the language and visual modalities interact throughout the videos. We propose a simple yet effective method to localize the narrated actions based on their expected duration. Through several experiments and analyses, we show that our method brings complementary information with respect to previous methods and leads to improvements over previous work for the task of temporal action localization.
WikiFactDiff is a dataset designed as a resource to perform atomic factual knowledge updates on language models, with the goal of aligning them with current knowledge. It describes the evolution of factual knowledge between two dates, named T_old and T_new, in the form of semantic triples. To enable the possibility of evaluating knowledge algorithms (such as ROME, MEND, MEMIT, etc.), these triples are verbalized and neighbor facts are determined to check for eventual bleedover.
WildQA is a video understanding dataset of videos recorded in outside settings. The dataset can be used to evaluate models for video question answering.
First of its kind paired win-fail action understanding dataset with samples from the following domains: “General Stunts,” “Internet Wins-Fails,” “Trick Shots,” & “Party Games.” The task is to identify successful and failed attempts at various activities. Unlike existing action recognition datasets, intra-class variation is high making the task challenging, yet feasible.
A benchmark + dataset for evaluating multimodal models on business process management (BPM) tasks. A dataset containing 2,928 human demonstrations of 598 workflows, each of which include a textual description of the workflow intent, a full screen recording of the human completing the workflow, an action trace detailing which screen elements were interacted with, and a manually-written step-by-step guide in the form of a "Standard Operating Procedure" (SOP). The demonstrations cover a wide range of tasks from 4 different websites interfaces sourced from the excellent WebArena benchmark. A benchmark with 6 novel BPM tasks that measure the ability of a model to generate accurate documentation, assist in knowledge transfer, and improve workflows. The tasks are designed to evaluate models on real-world applications of BPM tools, with a more broad focus than simple end-to-end automation. A fully automated evaluation harness for evaluating models on the introduced tasks. The harness is designed to be easy to use and can be run on any multimodal FM that can be prompted via API. The harness is open-source and available on our GitHub repository.
YouTube-Hands includes 240 videos which are annotated with hand trajectories.
YouwikiHow is a dataset for Weakly-Supervised temporal Article Grounding (WSAG). It contains 47K videos and an average of 20.8 query sentences for each video. Source: Weakly-Supervised Temporal Article Grounding Image Source: https://arxiv.org/pdf/2210.12444v1.pdf
This dataset focuses only on the robbery category, presenting a new weakly labelled dataset that contains 486 new real–world robbery surveillance videos acquired from public sources.
This dataset defines a total of 11 crowd motion patterns and it is composed of over 6000 video sequences with an average length of 100 frames per sequence. This documentation presents how to download and process the Crowd-11 dataset. If you use this dataset, please cite our paper: Camille Dupont, Luis Tobias, and Bertrand Luvison. "Crowd-11: A Dataset for Fine Grained Crowd Behaviour Analysis." In Computer Vision and Pattern Recognition Workshops (CVPRW), 2017. Since this dataset is a composition of web videos and already existing datasets, we ask you to download and accept licence of each source and dataset. The construction of the Crowd-11 dataset is composed of two steps: # Step 1: Retrieve videos of interest from the web and/or pre-existing datasets ## Retrieve the pre-existing datasets of interest The pre-existing datasets are: | DATASET NAME | url | $SOURCE_NAME | | ------------- |:-----------------------------------------------------------------:| -------------:| | UMN | http://mha.cs.umn.edu/proj_events.shtml#crowd | umn | | AGORASET |https://github.com/piallai/Agoraset | agoraset | | PETS | http://www.cvg.reading.ac.uk/PETS2009/a.html#s3 | pets | | HOCKEY FIGHT | http://visilab.etsii.uclm.es/personas/oscar/FightDetection/ | hockey | | MOVIES | http://visilab.etsii.uclm.es/personas/oscar/FightDetection/ | peliculas | | CUHK | http://www.ee.cuhk.edu.hk/~jshao/CUHKcrowd_files/cuhk_crowd_dataset.htm | cuhk | | WWW | http://www.ee.cuhk.edu.hk/~jshao/WWWCrowdDataset.html | www | | WORLDEXPO'10 CROWD COUNTING | http://www.ee.cuhk.edu.hk/~xgwang/expo.html | shanghai | | VIOLENT-FLOWS | http://www.openu.ac.il/home/hassner/data/violentflows/ | violent_flow | These datasets should be stored in their "existing_datasets/$SOURCE_NAME/" folder: . └── existing_datasets ├── agoraset ├── cuhk ├── hockey ├── peliculas │ ├── fights │ └── noFights ├── pets ├── shanghai ├── umn ├── violent_flow └── www ## Copy the videos of interest from the datasets of interest The list of the videos of interest is in existing_datasets_urls.csv. To extract them into the VOI folder, execute: python existing_datasets_gathering.py The VOI folder should have the following structure: . └── VOI ├── agoraset ├── cuhk ├── hockey ├── peliculas ├── pets ├── shanghai ├── umn ├── violent_flow └── www ## Download the videos of interest from the web The web sources are: | SOURCE NAME | url | $SOURCE_NAME | | ------------- |:----------------------------------:| -------------:| | YOUTUBE | https://www.youtube.com/ | youtube | | GETTYIMAGES | http://www.gettyimages.fr/ | gettyimages | | POND5 | https://www.pond5.com/ | pond5 | The list of the web urls to download is in web_urls.csv. The web_urls.csv file's structure is as follows : | $SOURCE NAME | URL | OUTPUT_NAME | TS_MULTIPLIER | | ------------- |:---------------------:| ------------:| --------------:| We do not provide the script to download them, but many tools exist to do it (pytube, urllib, etc...). Note: a few videos have a ts_multiplier field. These video are in slow motion and the ts_multiplier is provided to speed them up (cf. SETPTS option in avconv). The downloaded videos should be stored in their VOI/$SOURCE_NAME folder, which should now have the following structure: . └── VOI ├── agoraset ├── cuhk ├── gettyimages ├── hockey ├── peliculas ├── pets ├── pond5 ├── shanghai ├── umn ├── violent_flow ├── youtube └── www # Step 2: Processing original videos into the Crowd-11 dataset Once the VOI folder is complete, a preprocessing step is required in order to crop and trim the original videos into the Crowd-11 dataset. The preprocessing.csv file's structure is as follows : | Videoname | Label | Frame_start | Frame_end | Top_left | Top_right | Width | Height | $SOURCE_NAME | Scene_number | Crop_number | | ------------- |:---------:| ------------:| ----------:| ---------:| ----------:| -----:| ------:| ------------:| ------------:| -----------:| Installation: You need to have avconv installed: sudo apt-get install avconv Then, you need to install several python package. A virtualeenv installation is recommended: virtualenv -p python3 py source py/bin/activate pip install sk-video Execution (in the virtualenv): python script_formating.py
DAHLIA dataset [1] is devoted to human activity recognition, which is a major issue for adapting smart-home services such as user assistance. DAHLIA has been realized in Mobile Mii Platform by CEA LIST, and has been partly supported by ITEA 3 Emospaces Project (https://itea3.org/project/emospaces.html) Videos were recorded in realistic conditions, with 3 Kinect v2 sensors located as they would be in a real context. The long-range activities were performed in an unconstrained way (participants received only few instructions), and in a continuous (untrimmed) sequence, resulting in long videos (40 min in average per subject). Contrary to previously published databases, in which labeled actions are very short and have low-semantic level, this new database focuses on high-level semantic activities such as « Preparing lunch » or « House Working ». [1] G. Vaquette, A. Orcesi, L. Lucat and C. Achard, "The DAily Home LIfe Activity Dataset: A High Semantic Activity Dataset for Online Recognition," 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, pp. 497-504, doi: 10.1109/FG.2017.67.
The deliberate manipulation of public opinion, especially through altered images, poses a significant danger to society. To fight this issue on a technical level we support the research community by releasing the Digital Forensics 2023 (DF2023) training and validation dataset. The DF2023 training dataset comprises one million images from four major forgery categories: * splicing (400K) * copy-move (300K) * enhancement (200K) * removal (100K) This dataset enables an objective comparison of network architectures and can significantly reduce the time and effort of researchers preparing datasets. For a detailed description of the DF2023 dataset, please refer to: @inproceedings{Fischinger2023DFNet, title={DF2023: The Digital Forensics 2023 Dataset for Image Forgery Detection}, author={David Fischinger and Martin Boyer}, journal={The 25th Irish Machine Vision and Image Processing conference. (IMVIP)}, year={2023} } available from: Zenodo ## Naming convention The naming convention of DF2023 encodes information about the applied manipulations. Each image name has the following form: COCO_DF_0123456789_NNNNNNNN.{EXT} (e.g. COCO_DF_E000G40117_00200620.jpg) After the identifier of the image data source ("COCO") and the self-reference to the Digital Forensics ("DF") dataset, there are 10 digits as placeholders for the manipulation. Position 0 defines the manipulation types copy-move, splicing, removal, enhancement ([C,S,R,E]). The following digits 1-9 represent donor patch manipulations. For positions [1,2,7,8] (resample, flip, noise and brightness), a binary value indicates if this manipulation was applied to the donor image patch. Position 3 (rotate) indicates by the values 0-3 if the rotation was executed by 0, 90, 180 or 270 degrees. Position 4 defines if BoxBlur (B) or GaussianBlur (G) was used. Position 5 specifies the blurring radius. A value of 0 indicates that no blurring was executed. Position 6 indicates which of the Python-PIL contrast filters EDGE ENHANCE, EDGE ENHANCE MORE, SHARPEN, UnsharpMask or ImageEnhance (values 1-5) was applied. If none of them was applied, this value is set to 0. Finally, position 9 is set to the JPEG compression factor modulo 10, a value of 0 indicates that no JPEG compression was applied. The 8 characters NNNNNNNN in the image name template stand for a running number of the images.
This is an image splicing dataset including different types of preprocessing and postprocessing techniques. Foreground objects are taken from HRSOD and background images are taken from BG20k datasets. 95000 train and 5000 test images are provided.
InfiniteRep is a synthetic, open-source dataset for fitness and physical therapy (PT) applications. It includes 1k videos of diverse avatars performing multiple repetitions of common exercises. It includes significant variation in the environment, lighting conditions, avatar demographics, and movement trajectories. From cadence to kinematic trajectory, each rep is done slightly differently -- just like real humans. InfiniteRep videos are accompanied by a rich set of pixel-perfect labels and annotations, including frame-specific repetition counts. The dataset features: + 100 videos per exercise, spanning 5 to 10 repetitions each (1,000 videos total) + 7 unique indoor scenes + Realistic environmental occlusion (+ corresponding labels) + Diverse lighting conditions + Varied body shape, skin tones, and clothing + Rich annotations for 2D and 3D supervision ## Exercises The dataset currently includes the following exercises: + Pushups + Alternating Bicep Curls (with dumbbells) + Delt Flys (with dumbbells) + Squats + Bird Dogs + Supermans + Bicycle Crunches + Leg Raises + Front Raises (with dumbbells) + Overhead Press (with dumbbells) ## Annotations The dataset includes the following annotations: + Bounding boxes + Segmentation masks + Keypoints + Joint angles (quaternions) + Percent occlusion + Avatar characteristics + Camera position + and more Want depth labels? They are not included in the dataset but we can send them to you. Email us at info@toinfinity.ai. ## Download Download the dataset: toinfinity.ai/infiniterep Github repo with additional documentation: https://github.com/toinfinityai/InfiniteRep ## Need more data? Infinity AI specializes in generating custom synthetic data. If you need more (or different data), drop us a line at info@toinfinity.ai (we read every email).
Infinity AI's Spills Basic Dataset is a synthetic, open-source dataset for safety applications. It features 150 videos of photorealistic liquid spills across 15 common settings. Spills take on in-context reflections, caustics, and depth based on the surrounding environment, lighting, and floor. Each video contains a spill of unique properties (size, color, profile, and more) and is accompanied by pixel-perfect labels and annotations. This dataset can be used to develop computer vision algorithms to detect the location and type of spill from the perspective of a fixed camera. ## Key Features + 150 videos + 4 environments where spills commonly occur + 15 unique indoor scenes + Realistic spill appearance + Diverse variation in spill geometry + Multiple spill colors + Rich annotations ## Annotations Each video is accompanied by a rich set of pixel-perfect labels and annotations, including: + Spill characteristics (size, color, profile, depth, etc.) + Segmentation masks + Bounding boxes For the full description of labels and metadata, check out the README. ## File Size Total dataset size: 890MB All 150 videos are contained in a single zipped folder (560MB). Video resolutions (all mp4): 960 x 720 at 24fps 960 x 640 at 24fps 960 x 637 at 24fps 960 x 384 at 24fps. ## Resources + Github README: full dataset and annotation descriptions + Demo Jupyter notebook: Shows how to (1) filter/query the dataset based on metadata, and (2) how to visualize labels + Questions? We’re happy to chat asynchronously via email or hop on a call. Just send us a note at info@toinfinity.ai (this goes to all of the Infinity AI founders). ## Terms and Conditions This work is licensed under a Creative Commons Attribution 4.0 International License. Both academic and commercial applications are allowed. ## Build better models faster with synthetic data Infinity AI, the creator of this dataset, is a synthetic data company. You can download all of Infinity's synthetic datasets and APIs at the Infinity Marketplace. Feel free to get in touch in you want custom data, or want to book a scoping session on how to make synthetic data your competitive edge. www.infinity.ai | LinkedIn | Synthetic Data Blog
Laser powder bed fusion (LBPF) is the additive manufacturing (3D printing) process for metals. RAISE-LPBF is a large dataset on the effect of laser power and laser dot speed in 316L stainless steel bulk material. Both process parameters are independently sampled for each scan line from a continuous distribution, so interactions of different parameter choices can be investigated. Process monitoring comprises on-axis high-speed (20k FPS) video. The data can be used to derive statistical properties of LPBF, as well as to build anomaly detectors. RAISE-LPBF-Laser is the machine learning benchmark to reconstruct the laser parameters of the RAISE-LBPF dataset. Paper: https://doi.org/10.1016/j.addlet.2023.100161
A review on raw subjective scores and data manipulation for before and after refining Mean opinion Scores
It is released by the Shanghai Central Meteorological Observatory (SCMO) in 2020, records serval years of historical precipitation events in the Yangtze River delta area. The dataset contains a total of 43000 samples of precipitation events, of which 40000 samples for training and 3000 samples for testing. Each sample consists of 20 consecutive radar echo frames and lasts for 3 hours, where the first 10 frames are with intervals of 6 minutes and the last 10 frames are with intervals of 12 minutes. The echo frame has 460 * 460 resolution and covers 460km * 398km region. We additionally split out 3000 samples from the training set and use them for validation.
The University of Padova Body Pose Estimation dataset (UNIPD-BPE) is an extensive dataset for multi-sensor body pose estimation containing both single-person and multi-person sequences with up to 4 interacting people A network with 5 Microsoft Azure Kinect RGB-D cameras is exploited to record synchronized high-definition RGB and depth data of the scene from multiple viewpoints, as well as to estimate the subjects’ poses using the Azure Kinect Body Tracking SDK. Simultaneously, full-body Xsens MVN Awinda inertial suits allow obtaining accurate poses and anatomical joint angles, while also providing raw data from the 17 IMUs required by each suit. All the cameras and inertial suits are hardware synchronized, while the relative poses of each camera with respect to the inertial reference frame are calibrated before each sequence to ensure maximum overlap of the two sensing systems outputs. The setup used allowed to record synchronized 3D poses of the persons on the scene both via Xsens’ inverse kinematics algorithm (inertial motion capture) and by exploiting the Azure Kinect Body tracking SDK (markerless motion capture), simultaneously. The additional raw data (RGB, depth, camera network configuration) allow the user to assess the performance of any custom markerless motion capture algorithm (based on RGB, depth, or both). Further analyses can be progressed by varying the number of cameras being used and/or their resolution and frame rate. Moreover, raw angular velocities, linear accelerations, magnetic fields, and orientations from each IMU allow to develop and test multimodal BPE approaches focused on merging visual and inertial data. Finally, the precise body dimensions of each subject are provided. They include body height, weight, and segment lengths measured before the beginning of a recording session. They were used to scale the Xsens biomechanical model, and also constitute a ground truth for assessing the markerless BPE accuracy on estimating each subject’s body dimensions. The recorded sequences include 15 participants performing a set of 12 ADLs (e.g., walking, sitting, and jogging). The actions were chosen to present different challenges to BPE algorithms, including different movement speeds, self-occlusions, and complex body poses. Moreover, multi-person sequences, with up to 4 people performing a set of 7 different actions, are provided. Such sequences offer challenging scenarios where multiple self-occluded persons move and interact in a restricted space. They allow assessing the accuracy of multi-person tracking algorithms, focused on maintaining frame-by-frame consistent IDs of each detected person. A total of 13.3h of RGB, depth, and markerless BPE data are present in the dataset, corresponding to over 1,400,000 frames obtained from a calibrated network with 5 RGB-D cameras. The inertial suits, on the other hand, allowed to record 3h of inertial motion capture data, corresponding to a total of over 600,000 frames recorded by each of the 17 IMUs used by every suit.
The Storytelling Video Dataset is a high-quality, human-reviewed multimodal dataset featuring over 700 full-body video recordings of native Russian speakers. Each video is 10+ minutes long and includes synchronized speech, facial expressions, gestures, and emotional variation. The dataset is ideal for research and development in: 🗣️ Speech-to-text & voice modeling (Russian) 😃 Emotion & gesture recognition 🤖 Multimodal learning & LLM alignment 🧍 Avatar generation & digital human training Key Features: Full-body framing (waist-up or knee-up) Clear facial expressions and gestures High-quality speech audio in Russian Natural storytelling with emotional diversity Preview (10 Participants): We’ve prepared a short compilation of 10 different speakers from the dataset. Each clip is taken from a unique 10-minute unscripted video, featuring full-body framing, gestures, and emotional speech. 📺 Watch the sample: https://drive.google.com/drive/folders/1QtC2il-Qb62nZNJlOtG8WvOf-SSkSGM-?usp=drive_link – Video Preview Dataset Pages: 🐙 GitHub: github.com/MaratDV 📊 DataHub: datahub.io/@MaratDV/storytelling-video-dataset email chinzad@gmail.com or telegram @Marat_DV
We construct a fine-grained video-text dataset with 12K annotated high-resolution videos (~400k clips). The annotation of this dataset is inspired by the video script. If we want to make a video, we have to first write a script to organize how to shoot the scenes in the videos. To shoot a scene, we need to decide the content, shot type (medium shot, close-up, etc), and how the camera moves (panning, tilting, etc). Therefore, we extend video captioning to video scripting by annotating the videos in the format of video scripts. Different from the previous video-text datasets, we densely annotate the entire videos without discarding any scenes and each scene has a caption with ~145 words. Besides the vision modality, we transcribe the voice-over into text and put it along with the video title to give more background information for annotating the videos. In addition, we propose Vript-Bench, a new benchmark consisting of three challenging video understanding tasks (which are carefully double-checked by humans): - Vript-CAP (Caption): A benchmark with detailed captions rather than short captions. - Vript-RR (Retrieve then Reason): A video reasoning benchmark by first giving a detailed description of the scene as a hint and then asking questions about details in the scene. - Vript-ERO (Event Re-ordering): A benchmark that tests the temporal understanding by offering the descriptions of scenes located in two/four different timelines of the same video, and asks the model to give the right temporal order of the scenes.
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.