Fine-grained Evaluation of Dialog
FED 설명 보기
The FED dataset is constructed by annotating a set of human-system and human-human conversations with eighteen fine-grained dialog qualities.
데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.
The FED dataset is constructed by annotating a set of human-system and human-human conversations with eighteen fine-grained dialog qualities.
See paper: Caldas, Sebastian, et al. "Leaf: A benchmark for federated settings." arXiv preprint arXiv:1812.01097 (2018).
Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset
F-CelebA - This dataset is adapted from federated learning. Federated learning is an emerging machine learning paradigm with an emphasis on data privacy. The idea is to train through model aggregation rather than conventional data aggregation and keep local data staying on the local device. This dataset naturally consists of similar tasks and each of the 10 tasks contains images of a celebrity labeled by whether he/she is smiling or not. More detailed please check page https://github.com/ZixuanKe/CAT
ABSTRACT In this project, we propose a new comprehensive realistic cyber security dataset of IoT and IIoT applications, called Edge-IIoTset, which can be used by machine learning-based intrusion detection systems in two different modes, namely, centralized and federated learning. Specifically, the proposed testbed is organized into seven layers, including, Cloud Computing Layer, Network Functions Virtualization Layer, Blockchain Network Layer, Fog Computing Layer, Software-Defined Networking Layer, Edge Computing Layer, and IoT and IIoT Perception Layer. In each layer, we propose new emerging technologies that satisfy the key requirements of IoT and IIoT applications, such as, ThingsBoard IoT platform, OPNFV platform, Hyperledger Sawtooth, Digital twin, ONOS SDN controller, Mosquitto MQTT brokers, Modbus TCP/IP, ...etc. The IoT data are generated from various IoT devices (more than 10 types) such as Low-cost digital sensors for sensing temperature and humidity, Ultrasonic sensor, Water level detection sensor, pH Sensor Meter, Soil Moisture sensor, Heart Rate Sensor, Flame Sensor, ...etc.). However, we identify and analyze fourteen attacks related to IoT and IIoT connectivity protocols, which are categorized into five threats, including, DoS/DDoS attacks, Information gathering, Man in the middle attacks, Injection attacks, and Malware attacks. In addition, we extract features obtained from different sources, including alerts, system resources, logs, network traffic, and propose new 61 features with high correlations from 1176 found features. After processing and analyzing the proposed realistic cyber security dataset, we provide a primary exploratory data analysis and evaluate the performance of machine learning approaches (i.e., traditional machine learning as well as deep learning) in both centralized and federated learning modes. Instructions: Great news! The Edge-IIoT dataset has been featured as a "Document in the top 1% of Web of Science." This indicates that it is ranked within the top 1% of all publications indexed by the Web of Science (WoS) in terms of citations and impact. Please kindly visit kaggle link for the updates: https://www.kaggle.com/datasets/mohamedamineferrag/edgeiiotset-cyber-sec... Free use of the Edge-IIoTset dataset for academic research purposes is hereby granted in perpetuity. Use for commercial purposes is allowable after asking the leader author, Dr Mohamed Amine Ferrag, who has asserted his right under the Copyright. The details of the Edge-IIoT dataset were published in following the paper. For the academic/public use of these datasets, the authors have to cities the following paper: Mohamed Amine Ferrag, Othmane Friha, Djallel Hamouda, Leandros Maglaras, Helge Janicke, "Edge-IIoTset: A New Comprehensive Realistic Cyber Security Dataset of IoT and IIoT Applications for Centralized and Federated Learning", IEEE Access, April 2022 (IF: 3.37), DOI: 10.1109/ACCESS.2022.3165809 Link to paper : https://ieeexplore.ieee.org/document/9751703 ******************************************** The directories of the Edge-IIoTset dataset include the following: •File 1 (Normal traffic) -File 1.1 (Distance): This file includes two documents, namely, Distance.csv and Distance.pcap. The IoT sensor (Ultrasonic sensor) is used to capture the IoT data. -File 1.2 (Flame_Sensor): This file includes two documents, namely, Flame_Sensor.csv and Flame_Sensor.pcap. The IoT sensor (Flame Sensor) is used to capture the IoT data. -File 1.3 (Heart_Rate): This file includes two documents, namely, Flame_Sensor.csv and Flame_Sensor.pcap. The IoT sensor (Flame Sensor) is used to capture the IoT data. -File 1.4 (IR_Receiver): This file includes two documents, namely, IR_Receiver.csv and IR_Receiver.pcap. The IoT sensor (IR (Infrared) Receiver Sensor) is used to capture the IoT data. -File 1.5 (Modbus): This file includes two documents, namely, Modbus.csv and Modbus.pcap. The IoT sensor (Modbus Sensor) is used to capture the IoT data. -File 1.6 (phValue): This file includes two documents, namely, phValue.csv and phValue.pcap. The IoT sensor (pH-sensor PH-4502C) is used to capture the IoT data. -File 1.7 (Soil_Moisture): This file includes two documents, namely, Soil_Moisture.csv and Soil_Moisture.pcap. The IoT sensor (Soil Moisture Sensor v1.2) is used to capture the IoT data. -File 1.8 (Sound_Sensor): This file includes two documents, namely, Sound_Sensor.csv and Sound_Sensor.pcap. The IoT sensor (LM393 Sound Detection Sensor) is used to capture the IoT data. -File 1.9 (Temperature_and_Humidity): This file includes two documents, namely, Temperature_and_Humidity.csv and Temperature_and_Humidity.pcap. The IoT sensor (DHT11 Sensor) is used to capture the IoT data. -File 1.10 (Water_Level): This file includes two documents, namely, Water_Level.csv and Water_Level.pcap. The IoT sensor (Water sensor) is used to capture the IoT data. •File 2 (Attack traffic): -File 2.1 (Attack traffic (CSV files)): This file includes 13 documents, namely, Backdoor_attack.csv, DDoS_HTTP_Flood_attack.csv, DDoS_ICMP_Flood_attack.csv, DDoS_TCP_SYN_Flood_attack.csv, DDoS_UDP_Flood_attack.csv, MITM_attack.csv, OS_Fingerprinting_attack.csv, Password_attack.csv, Port_Scanning_attack.csv, Ransomware_attack.csv, SQL_injection_attack.csv, Uploading_attack.csv, Vulnerability_scanner_attack.csv, XSS_attack.csv. Each document is specific for each attack. -File 2.2 (Attack traffic (PCAP files)): This file includes 13 documents, namely, Backdoor_attack.pcap, DDoS_HTTP_Flood_attack.pcap, DDoS_ICMP_Flood_attack.pcap, DDoS_TCP_SYN_Flood_attack.pcap, DDoS_UDP_Flood_attack.pcap, MITM_attack.pcap, OS_Fingerprinting_attack.pcap, Password_attack.pcap, Port_Scanning_attack.pcap, Ransomware_attack.pcap, SQL_injection_attack.pcap, Uploading_attack.pcap, Vulnerability_scanner_attack.pcap, XSS_attack.pcap. Each document is specific for each attack. •File 3 (Selected dataset for ML and DL): -File 3.1 (DNN-EdgeIIoT-dataset): This file contains a selected dataset for the use of evaluating deep learning-based intrusion detection systems. -File 3.2 (ML-EdgeIIoT-dataset): This file contains a selected dataset for the use of evaluating traditional machine learning-based intrusion detection systems. ******************************************** Step 1: Downloading The Edge-IIoTset dataset From the Kaggle platform from google.colab import files !pip install -q kaggle files.upload() !mkdir ~/.kaggle !cp kaggle.json ~/.kaggle/ !chmod 600 ~/.kaggle/kaggle.json !kaggle datasets download -d mohamedamineferrag/edgeiiotset-cyber-security-dataset-of-iot-iiot -f "Edge-IIoTset dataset/Selected dataset for ML and DL/DNN-EdgeIIoT-dataset.csv" !unzip DNN-EdgeIIoT-dataset.csv.zip !rm DNN-EdgeIIoT-dataset.csv.zip Step 2: Reading the Datasets' CSV file to a Pandas DataFrame: import pandas as pd import numpy as np df = pd.read_csv('DNN-EdgeIIoT-dataset.csv', low_memory=False) Step 3 : Exploring some of the DataFrame's contents: df.head(5) print(df['Attack_type'].value_counts()) Step 4: Dropping data (Columns, duplicated rows, NAN, Null..): from sklearn.utils import shuffle drop_columns = ["frame.time", "ip.src_host", "ip.dst_host", "arp.src.proto_ipv4","arp.dst.proto_ipv4", "http.file_data","http.request.full_uri","icmp.transmit_timestamp", "http.request.uri.query", "tcp.options","tcp.payload","tcp.srcport", "tcp.dstport", "udp.port", "mqtt.msg"] df.drop(drop_columns, axis=1, inplace=True) df.dropna(axis=0, how='any', inplace=True) df.drop_duplicates(subset=None, keep="first", inplace=True) df = shuffle(df) df.isna().sum() print(df['Attack_type'].value_counts()) Step 5: Categorical data encoding (Dummy Encoding): import numpy as np from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler from sklearn import preprocessing def encode_text_dummy(df, name): dummies = pd.get_dummies(df[name]) for x in dummies.columns: dummy_name = f"{name}-{x}" df[dummy_name] = dummies[x] df.drop(name, axis=1, inplace=True) encode_text_dummy(df,'http.request.method') encode_text_dummy(df,'http.referer') encode_text_dummy(df,"http.request.version") encode_text_dummy(df,"dns.qry.name.len") encode_text_dummy(df,"mqtt.conack.flags") encode_text_dummy(df,"mqtt.protoname") encode_text_dummy(df,"mqtt.topic") Step 6: Creation of the preprocessed dataset df.to_csv('preprocessed_DNN.csv', encoding='utf-8') ******************************************** For more information about the dataset, please contact the lead author of this project, Dr Mohamed Amine Ferrag, on his email: mohamed.amine.ferrag@gmail.com More information about Dr. Mohamed Amine Ferrag is available at: https://www.linkedin.com/in/Mohamed-Amine-Ferrag https://dblp.uni-trier.de/pid/142/9937.html https://www.researchgate.net/profile/Mohamed_Amine_Ferrag https://scholar.google.fr/citations?user=IkPeqxMAAAAJ&hl=fr&oi=ao https://www.scopus.com/authid/detail.uri?authorId=56115001200 https://publons.com/researcher/1322865/mohamed-amine-ferrag/ https://orcid.org/0000-0002-0632-3172 Last Updated: 27 Mar. 2023
This dataset is derived from the Stack Overflow Data hosted by kaggle.com and available to query through Kernels using the BigQuery API: https://www.kaggle.com/stackoverflow/stackoverflow
This dataset contains 40,000 URLs of US federal environmental agency websites, along with links to captures in the Internet Archive Wayback Machine for 2016 and 2020 when present. It also contains the prevalence of 56 environmental terms and phrases and how the presence of those terms on the webpages changed from 2016 to 2020. During the Trump administration, website changes indicative of climate denial prompted civil society organizations to develop tools for tracking online government information sources. We examine a large sample of websites of US federal environmental agencies and show that between 2016 and 2020: 1) the use of the term “climate change” decreased by an estimated 38%; 2) access to as much as 20% of the Environmental Protection Agency’s website was removed; 3) changes were made more to Cabinet agencies’ websites and to highly visible pages. This dataset can be used to examine webpage change over time with the assistance of web archives.
We collect the various forms of Federal Reserve communications. - fomc_doc: FOMC post-meeting statements, Minutes, and press conferences. (122 docs) - speaker_doc: FOMC members’ speeches from over 30 different websites. (1300 docs) - period: Jan 2015 - Jul 2020 Open In Colab
FedTADBench is a federated time series anomaly detection benchmark. It covers 5 time series anomaly detection algorithms, 4 federated learning frameworks, and 3 time series anomaly detection datasets. Source: [FedTADBench: Federated Time-Series Anomaly Detection Benchmark (https://arxiv.org/pdf/2212.09518v1.pdf)
DOI: https://doi.org/10.7910/DVN/O4CRXK The most comprehensive standardised data on Malaysian federal and state elections from 1955 to the present. It draws from official election reports to provide well-curated panel datasets for federal, state, and by-elections. I thank the Election Commission of Malaysia for providing access to physical records of election results via Pusat Sumber SPR (Putrajaya), Nimesha Thevananthan for their assistance with data collection, as well as Danesh Prakash Chacko of Tindak Malaysia for providing valuable insights and assisting with filling gaps in the data, particularly for Sabah in 1971.
This dataset provides a collection of 162K images and 70 Videos of Meta-Humans. There are 10 Highly realistic Meta-Humans expressing 7 facial expressions.
A dataset of German parliament debates covering 74 years of plenary protocols across all 16 state parliaments of Germany as well as the German Bundestag. The debates are separated into individual speeches which are enriched with meta data identifying the speaker as a member of the parliament (mp). When using this data set, please cite the original paper "Lange, K.-R., Jentsch, C. (2023). SpeakGer: A meta-data enriched speech corpus of German state and federal parliaments. Proceedings of the 3rd Workshop on Computational Linguistics for Political Text Analysis@KONVENS 2023.". The meta data is separated into two different types: time-specific meta-data that contains only information for a legislative period but can change over time (e.g. the party or constituency of an mp) and meta-data that is considered fixed, such as the birth date or the name of a speaker. The former information are stored aong with the speeches as it is considered temporal information of that point in time, but are additionally stored in the file all_mps_mapping.csv if there is the need to double-check something. The rest of the meta-data are stored in the file all_mps_meta.csv. The meta-data from this file can be matched with a speech by comparing the speaker ID-variable "MPID". The speeches of each parliament are saved in a csv format. Along with the speeches, they contain the following meta-data: Period: int. The period in which the speech took place Session: int. The session in which the speech took place Chair: boolean. The information if the speaker was the chair of the plenary session Interjection: boolean. The information if the speech is a comment or an interjection from the crowd Party: list (e.g. ["cdu"] or ["cdu", "fdp"] when having more than one speaker during an interjection). List of the party of the speaker or the parties whom the comment/interjection references Consituency: string. The consituency of the speaker in the current legislative period MPID: int. The ID of the speaker, which can be used to get more meta-data from the file all_mps_meta.csv The file all_mps_meta.csv contains the following meta information: MPID: int. The ID of the speaker, which can be used to match the mp with his/her speeches. WikipediaLink: The Link to the mps Wikipedia page WikiDataLink: The Link to the mps WikiData page Name: string. The full name of the mp. Last Name: string. The last name of the mp, found on WikiData. If no last name is given on WikiData, the full name was heuristically cut at the last space to get the information neccessary for splitting the speeches. Born: string, format: YYYY-MM-DD. Birth date of the mp. If an exact birth date is found on WikiData, this exact date is used. Otherwise, a day in the year of birth given on Wikipedia is used. SexOrGender: string. Information on the sex or gender of the mp. Disclaimer: This infomation was taken from WikiData, which does not seem to differentiate between sex or gender. Occupation: list. Occupation(s) of the mp. Religion: string. Religious believes of the mp. AbgeordnetenwatchID: int. ID of the mp on the website Abgeordnetenwatch
This is the first federated quantum dataset in the literature.
Natural Vertical Partitioned CVR Dataset for Vertical Federated Learning This Dataset repo provides 2 industrial CVR Dataset for VFL research.
This file contains the data and code for the publication "The Federal Reserve's Response to the Global Financial Crisis and Its Long-Term Impact: An Interrupted Time-Series Natural Experimental Analysis" by A. C. Kamkoum, 2023.
The LfED-6D dataset is a collection of 6D grasp annotations acquired through experience (with a robot platform) or by human demonstration. For known objects, the annotated grasps can be directly applied given the pose of the object model is correctly computed. For unknown objects, the grasps can be generalized using methods for shape matching, for example the Dense Geometrical Correspondence Network. Source: LfED-6D