Feature Engineering

MuJoCo (multi-joint dynamics with contact) is a physics engine used to implement environments to benchmark Reinforcement Learning methods.

A labeled benchmark dataset for training machine learning models to statically detect malicious Windows portable executable files. The dataset includes features extracted from 1.1M binary files: 900K training samples (300K malicious, 300K benign, 300K unlabeled) and 200K test samples (100K malicious, 100K benign). Source: EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models

WebKB is a dataset that includes web pages from computer science departments of various universities. 4,518 web pages are categorized into 6 imbalanced categories (Student, Faculty, Staff, Department, Course, Project). Additionally there is Other miscellanea category that is not comparable to the rest. Source: Using Fuzzy Logic to Leverage HTML Markup for Web Page Representation

Data Set Information: Extraction was done by Barry Becker from the 1994 Census database. A set of reasonably clean records was extracted using the following conditions: ((AAGE>16) && (AGI>100) && (AFNLWGT>1)&& (HRSWK>0)) Prediction task is to determine whether a person makes over 50K a year.

UCI Machine Learning Repository is a collection of over 550 datasets.

The Sprites dataset contains 60 pixel color images of animated characters (sprites). There are 672 sprites, 500 for training, 100 for testing and 72 for validation. Each sprite has 20 animations and 178 images, so the full dataset has 120K images in total. There are many changes in the appearance of the sprites, they differ in their body shape, gender, hair, armor, arm type, greaves, and weapon. Source: Challenges in Disentangling Independent Factors of Variation

The KIT Motion-Language is a dataset linking human motion and natural language. Source: The KIT Motion-Language Dataset

METR-LA is a dataset for traffic prediction.

This dataset includes time-series data generated by accelerometer and gyroscope sensors (attitude, gravity, userAcceleration, and rotationRate). It is collected with an iPhone 6s kept in the participant's front pocket using SensingKit which collects information from Core Motion framework on iOS devices. All data is collected in 50Hz sample rate. A total of 24 participants in a range of gender, age, weight, and height performed 6 activities in 15 trials in the same environment and conditions: downstairs, upstairs, walking, jogging, sitting, and standing. Source: https://github.com/mmalekzadeh/motion-sense Image Source: https://github.com/mmalekzadeh/motion-sense

Abstract: Measurements of electric power consumption in one household with a one-minute sampling rate over a period of almost 4 years. Different electrical quantities and some sub-metering values are available. | Data Set Characteristics | Number of Instances | Area | Attribute Characteristics | Number of Attributes | Date Donated | Associated Tasks | Missing Values | | ------------------------- | ------------------- | -------- | ------------------------- | -------------------- | ------------ | ---------------------- | -------------- | | Multivariate, Time-Series | 2075259 | Physical | Real | 9 | 2012-08-30 | Regression, Clustering | Yes | ### Source: Georges Hebrail (georges.hebrail '@' edf.fr), Senior Researcher, EDF R&D, Clamart, France Alice Berard, TELECOM ParisTech Master of Engineering Internship at EDF R&D, Clamart, France ### Data Set Information: This archive contains 2075259 measurements gathered in a house located in Sceaux (7km of Paris, France) between December 2006 and November 2010 (47 months). Notes: 1. (global_active_power*1000/60 - sub_metering_1 - sub_metering_2 - sub_metering_3) represents the active energy consumed every minute (in watt hour) in the household by electrical equipment not measured in sub-meterings 1, 2 and 3. 2. The dataset contains some missing values in the measurements (nearly 1,25% of the rows). All calendar timestamps are present in the dataset but for some timestamps, the measurement values are missing: a missing value is represented by the absence of value between two consecutive semi-colon attribute separators. For instance, the dataset shows missing values on April 28, 2007. ### Attribute Information: 1. date: Date in format dd/mm/yyyy 2. time: time in format hh:mm:ss 3. global_active_power: household global minute-averaged active power (in kilowatt) 4. global_reactive_power: household global minute-averaged reactive power (in kilowatt) 5. voltage: minute-averaged voltage (in volt) 6. global_intensity: household global minute-averaged current intensity (in ampere) 7. sub_metering_1: energy sub-metering No. 1 (in watt-hour of active energy). It corresponds to the kitchen, containing mainly a dishwasher, an oven and a microwave (hot plates are not electric but gas powered). 8. sub_metering_2: energy sub-metering No. 2 (in watt-hour of active energy). It corresponds to the laundry room, containing a washing-machine, a tumble-drier, a refrigerator and a light. 9. sub_metering_3: energy sub-metering No. 3 (in watt-hour of active energy). It corresponds to an electric water-heater and an air-conditioner. ### Relevant Papers: N/A ### Citation Request: This dataset is made available under the “Creative Commons Attribution 4.0 International (CC BY 4.0)” license

Useful for through two applications - automatic readability assessment and automatic text simplification. The corpus consists of 189 texts, each in three versions (567 in total). Source: OneStopEnglish corpus: A new corpus for automatic readability assessment and text simplification

The PhysioNet Challenge 2012 dataset is publicly available and contains the de-identified records of 8000 patients in Intensive Care Units (ICU). Each record consists of roughly 48 hours of multivariate time series data with up to 37 features recorded at various times from the patients during their stay such as respiratory rate, glucose etc. Source: Multi-resolution Networks For Flexible Irregular Time Series Modeling (Multi-FIT) Image Source: https://physionet.org/content/challenge-2016/1.0.0/

A novel large dataset of social media posts from users with one or multiple mental health conditions along with matched control users. Source: SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions

This data set includes hourly air pollutants data from 12 nationally-controlled air-quality monitoring sites. The air-quality data are from the Beijing Municipal Environmental Monitoring Center. The meteorological data in each air-quality site are matched with the nearest weather station from the China Meteorological Administration. The time period is from March 1st, 2013 to February 28th, 2017. Missing data are denoted as NA.

NCBI Datasets is a valuable resource that simplifies the process of gathering data from various NCBI databases. Whether you’re a researcher, scientist, or bioinformatician, NCBI Datasets provides an efficient way to access sequence information, annotations, and metadata for genes and genomes.

The Ecoli dataset is a dataset for protein localization. It contains 336 E.coli proteins split into 8 different classes.

A diverse dataset of human faces, including unconventional poses, occluded faces, and a vast variability in backgrounds. Source: DeepPrivacy: A Generative Adversarial Network for Face Anonymization

Unified Time Series Dataset (UTSD) includes 7 domains with up to 1 billion time points with hierarchical capacities to facilitate research of large models in the field of time series. It is meticulously assembled from a blend of publicly accessible online data repositories and empirical data derived from real-world machine operations. We analyze each dataset within the collection, examining the time series through the lenses of stationarity and forecastability to allows us to characterize the level of complexity inherent to each dataset. All datasets are classified into seven distinct domains by their source: Energy, Environment, Health, Internet of Things (IoT), Nature, Transportation, and Web with diverse sampling frequencies. UTSD is constructed with hierarchical capacities, namely UTSD-1G, UTSD-2G, UTSD-4G, and UTSD-12G, where each smaller dataset is a subset of the larger ones. A larger subset means greater data difficulty and diversity, allowing you to conduct detailed scaling experiments. See the paper and codebase for more information.

The original dataset from Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting contains traffic readings collected from 207 loop detectors on highways in Los Angeles County, aggregated in 5 minutes intervals over four months between March 2012 and June 2012. The Point missing setting, introduced in Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural Networks, is a variant for imputation in which 25% of data are masked out uniformly at random. Results on this dataset are assumed to be obtained in-sample, meaning that the test interval is used also for training, excluding data used for evaluation.

The original dataset from Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting contains 6 months of traffic readings from 01/01/2017 to 05/31/2017 collected every 5 minutes by 325 traffic sensors in San Francisco Bay Area. The measurements are provided by California Transportation Agencies (CalTrans) Performance Measurement System (PeMS). The Point missing setting, introduced in Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural Networks, is a variant for imputation in which 25% of data are masked out uniformly at random. Results on this dataset are assumed to be obtained in-sample, meaning that the test interval is used also for training, excluding data used for evaluation.

PulseImpute is a benchmark for Pulsative Physiological Signal Imputation which includes realistic mHealth missingness models, an extensive set of baselines, and clinically-relevant downstream tasks. It contains 440,953 100 Hz 5-minute ECG waveforms from 32,930 patients Source: PulseImpute: A Novel Benchmark Task for Pulsative Physiological Signal Imputation Image Source: https://arxiv.org/pdf/2212.07514v1.pdf

The AIDS Antiviral Screen dataset is a dataset of screens checking tens of thousands of compounds for evidence of anti-HIV activity. The available screen results are chemical graph-structured data of these various compounds. Source: Graph Neural Processes Towards Bayesian Graph Neural Networks