Papers with Code 아카이브

Anomaly Detection

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
28
평가표
76
보존된 평가 기록
669

Anomaly Detection is a binary classification identifying unusual or unexpected patterns in a dataset, which deviate significantly from the majority of the data. The goal of anomaly detection is to identify such anomalies, which could represent errors, fraud, or other types of unusual events, and flag them for further investigation.

[Image source]: GAN-based Anomaly Detection in Imbalance Problems

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

Hazards&Robots

We consider the problem of detecting, in the visual sensing data stream of an autonomous mobile robot, semantic patterns that are unusual (i.e., anomalous) with respect to the robot’s previous experience in similar environments. These anomalies might indicate unforeseen hazards and, in scenarios where failure is costly, can be used to trigger an avoidance behavior. We contribute three novel image-based datasets acquired in robot exploration scenarios, comprising a total of more than 200k labeled frames, spanning various types of anomalies.

SensumSODF

Given the unavailability of real-world pharmaceutical inspection-domain datasets, we have created the Sensum Solid Oral Dosage Forms (SensumSODF) dataset intended for research and evaluation purposes. The dataset consists of two main types of solid oral dosage forms: - capsule; non-translucent hard-shelled capsules with print, which are normally used for dry, powdered ingredients - softgel; translucent soft-shelled capsules, which are primarily used for oils and active ingredients which are dissolved or suspended in oil The dataset consists of 836 defect-free and 153 defective examples of non-translucent hard-shelled capsules with a size of 192 × 320 pixels, and 846 defect-free and 345 defective examples of translucent soft-shelled capsules with a size of 144 × 144 pixels. Defective examples exhibit diverse defects such as cracks, dents, smudges, impurities, and air bubbles. Defective regions range from small to large structures and are hand-annotated by a pharmaceutical product inspection domain-expert. Annotations coarsely indicate the defective area on a given example, yet to some extent also include minor defect-free areas. Dataset paper: Detection of surface defects on pharmaceutical solid oral dosage forms with convolutional neural networks

SKAB

SKAB is designed for evaluating algorithms for anomaly detection. The benchmark currently includes 30+ datasets plus Python modules for algorithms’ evaluation. Each dataset represents a multivariate time series collected from the sensors installed on the testbed. All instances are labeled for evaluating the results of solving outlier detection and changepoint detection problems.

Stream-51

A new dataset for streaming classification consisting of temporally correlated images from 51 distinct object categories and additional evaluation classes outside of the training distribution to test novelty recognition. Source: Stream-51: Streaming Classification and Novelty Detection from Videos

SupplyGraph

Graph Neural Networks (GNNs) have gained traction across different domains such as transportation, bio-informatics, language processing, and computer vision. However, there is a noticeable absence of research on applying GNNs to supply chain networks. Supply chain networks are inherently graphlike in structure, making them prime candidates for applying GNN methodologies. This opens up a world of possibilities for optimizing, predicting, and solving even the most complex supply chain problems. A major setback in this approach lies in the absence of real-world benchmark datasets to facilitate the research and resolution of supply chain problem using GNNs. To address the issue, we present a real-world benchmark dataset for temporal tasks, obtained from one of the leading FMCG companies in Bangladesh, focusing on supply chain planning for production purposes. The dataset includes temporal data as node features to enable sales predictions, production planning, and the identification of factory issues. By utilizing this dataset, researchers can employ GNNs to address numerous supply chain problems, thereby advancing the field of supply chain analytics and planning. - Dataset GitHub - arXiv - PDF on arXiv Read the paper to learn more details and data statistics.

TIMo

TIMo (Time-of-Flight Indoor Monitoring) is a dataset of infrared and depth videos intended for the use in Anomaly Detection and Person Detection/People Counting. It features more than 1,500 sequences for anomaly detection, which sum up to more than 500,000 individual frames. For person detection the dataset contains more than than 240 sequences. The data was captured using a Microsoft Azure Kinect RGB-D camera. In addition, we provide annotations of anomalous frame ranges for use with anomaly detection and bounding boxes and segmentation masks for use with person detection. The data was captured in parts from a tilted view and a top-down perspective.

WFDD

WFDD is a dataset for benchmarking anomaly detection methods with a focus on textile inspection. It includes 4101 woven fabric images categorized into 4 categories: grey cloth, grid cloth, yellow cloth, and pink flower. The first three classes are collected from the industrial production sites of WEIQIAO Textile, while the 'pink flower' class is gathered from the publicly available Cloth Flaw Dataset. Each category contains block-shape, point-like, and line-type defects with pixel-level annotations.

Yahoo S5

Automatic anomaly detection is critical in today's world where the sheer volume of data makes it impossible to tag outliers manually. The goal of this dataset is to benchmark your anomaly detection algorithm. The dataset consists of real and synthetic time-series with tagged anomaly points. The dataset tests the detection accuracy of various anomaly-types including outliers and change-points. The synthetic dataset consists of time-series with varying trend, noise and seasonality. The real dataset consists of time-series representing the metrics of various Yahoo services.

COMPASS-XP

COMPASS-XP is a dataset of matched photographic and X-ray images of single objects, made available for use in Machine Learning & Computer Vision research, in particular in the context of transport security. Objects are imaged in multiple poses, and accompanied by metadata including labels for whether we consider the object to be dangerous in the context of aviation. Object classes overlap with those in the popular ImageNet Large Scale Visual Recognition Challenge class set and theWordNet lexical database, and identifiers for shared classes in both schemes are also provided. Hardware Configuration Photographs were captured with a Sony DSC-W800 compact digital camera. X-ray scans were obtained using a Gilardoni FEP ME 536 mailroom X-ray machine, distributed in the UK by Todd Research under the name TR50. The scanner is dual energy and generates several image outputs: • Low: Raw 8-bit greyscale data from the scanner’s low energy X-ray channel. • High: Raw 8-bit greyscale data from the scanner’s high energy X-ray channel. • Density: 8-bit greyscale data representing inferred material density computed from the two channels. • Grey: RGB PNG image representing a combination of both low and high energy channels with some appearance improvements. Although nominally greyscale, the image does include subtle duotone-style colouration. • Colour RGB PNG image with false colour palette representing material density. In practice the grey and colour versions are probably most useful, but for completeness the dataset includes all variants for each scan. Data Files Image files are supplied in six subdirectories, corresponding to the five X-ray image variants above plus photos. X-rays are provided in PNG format, while photos are JPEG. Each scan is identified by a numeric index, which is also used to name the file, padded with leading zeros to always be 4 digits long. Scan metadata is provided in the accompanying tab-delimited text file, meta.txt. This includes the following columns: • basename: The zero-padded identifier for the scan. All six image type variants for the same class-instance-pose have the same basename. X-ray files are named basename.png while photos are basename.jpg. • class: The object class in the scan. • instance: An integer identifying the object instance. Instances start at 1 for each class. • pose: An integer identifying the object pose. Poses start at 1 for each instance. • scan tray: Either A, indicating that the pose was imaged in a weighted tray, or N indicating it was not. • dangerous: Whether the object was considered dangerous (True/False). • IN id: Numeric index of the object class in the ILSVRC list of 1000 classes, or empty if the class isn’t present there. • WN id: WordNet identifier for the object class, or empty if the class isn’t present inWordNet. License The COMPASS-XP dataset was acquired as part of a research project funded by the UK Government Future Aviation Security Solutions programme. Both the images and their metadata are licensed under the Creative Commons Attribution 4.0 International License and may be freely used for research and commercial purpose, including derivative works, providing the source is acknowledged. COMPASS-XP Dataset Authors Lewis D. Griffin*, Matthew Caldwell, Jerone T. A. Andrews Computational Security Science Group, UCL * l.griffin@cs.ucl.ac.uk

edeniss2020

Overview The edeniss2020 dataset is a time series dataset. It consists of equidistant sensor readings stemming from 97 sensors in the EDEN ISS research greenhouse. EDEN ISS was a (almost) closed loop research greenhouse build under the lead of the German Aerospace Center to study Controlled Environment Agriculture (CEA) techniques and plant growth for future long-term space missions. EDEN ISS was deployed in Antarctica in 2018 next to the german Neumayer III polar station and has been in operation for four years. The data contained in the edeniss2020 dataset was recorded during the third mission year in 2020 between 2020/01/01 and 2020/12/30. Every sensor within the dataset is related to one of the following Subsystems |Acronym | Description | |--|--| | AMS‑FEG | Atmosphere Management System (AMS) of the Future Exploration Greenhouse (FEG) | | AMS‑SES | Atmosphere Management System AMS of the Service Section (SES)| | ICS | Illumination Control System | | NDS | Nutrient Delivery System | | TCS | Thermal Control System| ## Specification |Item|Description | |--|--| | Number of Files | 97 | | Start Date | 2020/01/01 00:00:05 | | End Date | 2020/12/30 23:55:00 | | Sampling Rate | 1/300 Hz (5min) | ## Contents The dataset includes the following files: - ams-feg/*.csv: Sensor readings related to the AMS-FEG - ams-ses/*.csv: Sensor readings related to the AMS-SES - ics/*.csv: Temperature readings related to the ICS - nds/*.csv: Sensor readings related to the NDS - tcs/*.csv: Sensor readings related to the - edeniss2020.csv: Description and Units of the measurements. - README.md: This file. | Subsystem | Sensor | #sensors | Note | |--|--|--|--| | AMS-FEG | CO2 | 2 | | | | Photosynthetic Active Radiation (PAR) | 2 | | | | Relative Humidity (RH) | 2 | | | | Temperature (T) | 3 | | | AMS-SES | CO2 | 2 | | | Photosynthetic Active Radiation (PAR) | 1 | | | | Relative Humidity (RH) | 1 | | | | Temperature (T) | 3 | | | | Vapor Pressure Deficit (VPD) | 1 | | | ICS | Temperature (T) | 38 | Measured at the LED lamp above each growth tray | | | NDS | Electrical Conductivity (EC) | 4 | EC of the nutrient solutions | | | Level (H) | 2 | Level of the solution in the nutrient solution tanks | | | PH-Value (PH) | 4 | PH Value of the nutrient solutions | | | Pressure \(P\) | 8 | Pressure in the piping from the tanks to the growth racks in the FEG | | | Temperature (T) | 4 | Temperature of the nutrient solutions. | | | Volume (V) | 2 | Volume up to the level sensor | | TCS | Pressure \(P\) | 3 | | | Relative Humidity (RH) | 2 | | | Temperature (T) | 12 | | | Valve (VALVE) | 3 | ## Usage The dataset can be used for uni- and multivariate analysis. To read a uni-variate time series with pandas do: python import pandas as pd # Example of loading ams-feg/co2-1.csv data = pd.read_csv('./ams-feg/co2-1.csv') print(data.head()) To read the data for a whole subsystem (e.g. AMS-FEG) as a multivariate time series: python import pandas as pd import glob SUBSYSTEM = 'ams-feg' combined_df = pd.DataFrame() for i, file in enumerate(glob.glob(f'./{SUBSYSTEM}/*.csv')): df = pd.read_csv(file, header=0, usecols=[0, 1]) if i == 0: combined_df = df else: combined_df = pd.merge(combined_df, df, on='time') combined_df = combined_df.sort_values(by='time').reset_index(drop=True) print(combined_df) ## License Creative Commons Attribution 4.0 International (CC BY 4.0 Deed) ## Citation If you use this dataset, please cite it as follows: Rewicki, F., Norman, T., Vrakking, V., (2024). edeniss2020. Zenodo. http://doi.org/10.5281/zenodo.11485183 ## Contact Ferdinand Rewicki: ferdinand.rewicki@dlr.de ## Version History * v1.0 Initial Version

Failure-Dataset-OpenStack

This failure dataset contains information on the events collected in the OpenStack cloud computing platform during three different campaigns of fault-injection experiments performed with three different workloads.

HERA RFI Detection

This dataset contains simulated and expert-labelled spectrograms from two radio telescopes: the Hydrogen Epoch of Reionization Array (HERA) in South Africa and the Low-Frequency Array (LOFAR) in the Netherlands. These datasets are intended to test radio-frequency interference (RFI) detection schemes. This entry pertains to the HERA dataset specifically. The HERA dataset is comprised of: - 560 Spectrograms and accompanying binary mask information. - Several copies of this dataset with classes of RFI sources removed for out-of-distribution testing. This dataset has been preprocessed into 512 x 512 x 1 images. The low number of target pixels (RFI) proportional to the total data volume makes this dataset a useful test for RFI detection schemes in radio astronomy and, more generally, anomaly detection or semantic segmentation methods.

Icons-50

Icons-50 is a dataset for studying surface variation robustness. Source: Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations

IMS Bearing Dataset

Bearing acceleration data from three run-to-failure experiments on a loaded shaft. The data set was provided by the Center for Intelligent Maintenance Systems (IMS), University of Cincinnati.

Insider Threat Test Dataset

The Insider Threat Test Dataset is a collection of synthetic insider threat test datasets that provide both background and malicious actor synthetic data. The CERT Division, in partnership with ExactData, LLC, and under sponsorship from DARPA I2O, generated a collection of synthetic insider threat test datasets. These datasets provide both synthetic background data and data from synthetic malicious actors. For more background on this data, please see the paper, Bridging the Gap: A Pragmatic Approach to Generating Insider Threat Data. Datasets are organized according to the data generator release that created them. Most releases include multiple datasets (e.g., r3.1 and r3.2). Generally, later releases include a superset of the data generation functionality of earlier releases. Each dataset file contains a readme file that provides detailed notes about the features of that release. The answer key file answers.tar.bz2 contains the details of the malicious activity included in each dataset, including descriptions of the scenarios enacted and the identifiers of the synthetic users involved.

Large-scale Anomaly Detection

Large-scale Anomaly Detection (LAD) is a database to benchmark anomaly detection in video sequences, which is featured in two aspects. 1) It contains 2000 video sequences including normal and abnormal video clips with 14 anomaly categories including crash, fire, violence, etc. with large scene varieties, making it the largest anomaly analysis database to date. 2) It provides the annotation data, including video-level labels (abnormal/normal video, anomaly type) and frame-level labels (abnormal/normal video frame) to facilitate anomaly detection.

LOFAR RFI Detection

This dataset contains simulated and expert-labelled spectrograms from two radio telescopes: the Hydrogen Epoch of Reionization Array (HERA) in South Africa and the Low-Frequency Array (LOFAR) in the Netherlands. These datasets are intended to test radio-frequency interference (RFI) detection schemes. This entry pertains to the LOFAR dataset specifically. The LOFAR dataset is comprised of: - 7609 Spectrograms and accompanying binary mask information. This dataset has been preprocessed into 512 x 512 x 1 images. The low number of target pixels (RFI) proportional to the total data volume makes this dataset a useful test for RFI detection schemes in radio astronomy and, more generally, anomaly detection or semantic segmentation methods.

NovelCraft

Scene-focused, multi-modal, episodic data of the images and symbolic world-states seen by an agent completing a pogo-stick assembly task within a video game world. Classes consist of episodes with novel objects inserted. A subset of these novel objects can impact gameplay and agent behavior. Novelty objects can vary in size, position, and occlusion within the images. Usable for novelty detection, generalized category discovery, and class-imbalanced classification.

PAD Dataset

Multi-pose Anomaly Detection (MAD) dataset, which represents the first attempt to evaluate the performance of pose-agnostic anomaly detection. The MAD dataset containing 4,000+ highresolution multi-pose views RGB images with camera/pose information of 20 shape-complexed LEGO animal toys for training, as well as 7,000+ simulation and real-world collected RGB images (without camera/pose information) with pixel-precise ground truth annotations for three types of anomalies in test sets. Note that MAD has been further divided into MAD-Sim and MAD-Real for simulation-to-reality studies to bridge the gap between academic research and the demands of industrial manufacturing.

PHANTOM

To evaluate the presented approaches, we created the Physical Anomalous Trajectory or Motion (PHANTOM) dataset consisting of six classes featuring everyday objects or physical setups, and showing nine different kinds of anomalies. We designed our classes to evaluate detection of various modes of video abnormalities that are generally excluded in video AD settings. The train and test sets of each class contain approximately 30 videos of varying lengths. The train set contains only normal videos, while the test set is evenly balanced between normal and anomalous videos. The classes were designed to be of varying difficulties and to feature different types of anomalies. For example, the window class was filmed in multiple lighting scenarios to increase variance. The normal videos include motion that follows an expected trajectory (pendulum, keyboard) or an expected movement (window). The sushi class features procedural motion, while candle and magnets feature more subtle motion that only appears locally. The anomalous videos can feature an interference of the regular motion (window, candle, magnets), an added or removed step in the usual procedure (sushi), motion that follows a different trajectory (pendulum, keyboard), or contains a different object (pendulum).

TEP

The original paper presented a model of the industrial chemical process named Tennessee Eastman Process and a model-based TEP simulator for data generation. The most widely used benchmark consists of 22 datasets, 21 of which (Fault 1–21) contain faults and 1 (Fault 0) is fault-free. It is available in repository. All datasets have training (500 samples) and testing (960 samples) parts: training part has healthy state observations, testing part begins right after training, and contains faults which appear after 8 h since the training part. Each dataset has 52 features or observation variables with a 3 min sampling rate for most of all. Source: Unsupervised Offline Changepoint Detection Ensembles

TII-SSRC-23

The TII-SSRC-23 dataset offers a comprehensive collection of network traffic patterns, meticulously compiled to support the development and research of Intrusion Detection Systems (IDS). It presents a dual structure: one part provides a tabular representation of extracted features in CSV format, while the other offers raw network traffic data for each type of traffic in PCAP files. This rich dataset captures both benign and malicious network scenarios, serving as an invaluable resource for researchers in the machine learning field. URL: https://www.kaggle.com/datasets/daniaherzalla/tii-ssrc-23

Vehicle Claims

The code to create the dataset is available here. The dataset used in the paper is available on github - Maker - Categorical - The brand of the vehicle. - GenModel - Categorical - The model of the vehicle. - Color - Categorical - Colour of the vehicle. - Reg_Year - Categorical - Year of Registration. - Body_Type - Categorical - Eg. SUV, Convertible. - Runned_Miles - Numerical - Distance covered by the vehicle. - Engin_Size - Categorical - Size of engine. - GearBox - Categorical - Automatic, Manual. - FuelType - Categorical - Petrol, Diesel. - Price - Numerical - Price of vehicle. - Seat_num - Numerical - Number of seats. - Door_num - Numerical - Number of Doors. - issue - Categorical - Type of damage. - issue_id - Categorical - Specific damage. - repair_complexity - Categorical - Difficulty to repair the vehicle. - repair_hours - Numerical - Time required to finish the job. - repair_cost - Numerical - Cost of repair. Other attributes are not used for evaluation in this work. breakdown_date and repair_date were added with the idea of inserting anomalies based on the number of days required to repair the vehicle.

VFP290K

Vision-based Fallen Person (VFP290K) is a novel, large-scale dataset for the detection of fallen persons composed of fallen person images collected in various real-world scenarios. VFP290K consists of 294,714 frames of fallen persons extracted from 178 videos, including 131 scenes in 49 locations.

5GAD-2022

This dataset contains two types of intercepted network packets: "normal" network traffic packets (i.e. a variety of non-malicious traffic types) and "attack" packets from attacks against a 5G Core implemented with free5GC. The captures were collected using tshark or Wireshark on 4 different network interfaces within the 5G core. Those interfaces and where they sit within the system are outlined in the 5GNetworkDiagram figure. Files that start with "allcap" contain packets that were recorded on all four interfaces simultaneously; other *.pcapng files represent the same data that has been broken out into one of the four interfaces.

ALLO

ALLO is an anomaly detection and localization dataset for space stations in lunar orbit. Synthetically rendered using Blender, ALLO provides realistic images of what a robotic manipulator on a space station will encounter including possible anomalies.

Bosch CNC Machining Dataset

The dataset provided is a collection of real-world industrial vibration data collected from a brownfield CNC milling machine. The acceleration has been measured using a tri-axial accelerometer (Bosch CISS Sensor) mounted inside the machine. The X- Y- and Z-axes of the accelerometer have been recorded using a sampling rate equal to 2 kHz. Thereby normal as well as anomalous data have been collected for 4 different timeframes, each lasting 5 months from February 2019 until August 2021 and labelled accordingly. It can be used to investigate the scalability of models and research process variations as the anomaly impact differs. In total there is data from three different CNC milling machines each executing 15 processes. For a detailed description of the data and experimental set-up, please refer to the paper: https://doi.org/10.1016/j.procir.2022.04.022

BottleCap

The BottleCap dataset contains over 1100 color images and 7 types of real defects. All the images are collected from the real production line and verified by professionals.

COCO-OOC

COCO-OOC goes beyond standard object detection to ask the question: Which objects are out-of-context (OOC)? Given an image with a set of objects, the goal of COCO-OOC is to determine if an object is inconsistent with the contextual relations, where it must detect the OOC object with a bounding box. COCO-OOC is derived from COCO by inserting objects into images that violate contextual relationships compared to the existing objects in a scene. COCO-OOC has 106,036 images with two kinds of OOC violations: co-occurrence and size.

Domain-independent anomalies datasets

An adaption of the MVTec Anomaly Detection dataset, presented in the paper "Domain-independent detection of known anomalies". There are three different datasets, each covering one specific anomaly type: color, cut and hole. The datasets can be used to evaluate approaches on the hybrid task of detecting known anomalies across different, previously unseen objects: All object types except one are used for training. During testing, the images of the remaining object type should be classified on whether they contain an anomaly. Note: the authors are not affiliated with MVTec

ESA-AD

ESA Anomaly Dataset is the first large-scale, real-life satellite telemetry dataset with curated anomaly annotations originated from three ESA missions. We hope that this unique dataset will allow researchers and scientists from academia, research institutes, national and international space agencies, and industry to benchmark models and approaches on a common baseline as well as research and develop novel, computational-efficient approaches for anomaly detection in satellite telemetry data. The dataset results from the work of an 18-month project carried by an industry Consortium composed of Airbus Defence and Space, KP Labs and the European Space Agency’s European Space Operations Centre. The project, funded by the European Space Agency (ESA), is part of the Artificial Intelligence for Automation (A²I) Roadmap (De Canio et al., 2023), a large endeavour started in 2021 to automate space operations by leveraging artificial intelligence.

FedTADBench

FedTADBench is a federated time series anomaly detection benchmark. It covers 5 time series anomaly detection algorithms, 4 federated learning frameworks, and 3 time series anomaly detection datasets. Source: [FedTADBench: Federated Time-Series Anomaly Detection Benchmark (https://arxiv.org/pdf/2212.09518v1.pdf)

GADformer Trajectory Datasets

This dataset collection provides trajectory datasets of - synthetic trajectories with variants of different noise and novelty levels - brightkite with labels for evaluation - amazon drivers with labels for evaluation - Deutsche Bahn DBCargo shipcontainer trajectories with labels for evaluation

Hawk Annotation Dataset

Hawk Annotation Dataset includes language descriptions specifically for anomaly scenes in seven existing video anomaly datasets. These seven datasets include a variety of anomalous scenarios such as crime (UCF-Cirme), campus (ShanghaiTech and CUHK Avenue), pedestrian walkways (UCSD Ped1 and Ped2), traffic (DoTA), and human behavior (UBnormal). With the support of these visual scenarios, this dataset can perform comprehensive fine-tuning for various abnormal scenarios, being closer to open-world scenarios.

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.