General Language Understanding Evaluation (GLUE) benchmark is a collection of nine natural language understanding tasks, including single-sentence tasks CoLA and SST-2, similarity and paraphrasing tasks MRPC, STS-B and QQP, and natural language inference tasks MNLI, QNLI, RTE and WNLI. Source: Align, Mask and Select: A Simple Method for Incorporating Commonsense Knowledge into Language Representation Models Image Source: https://gluebenchmark.com/

The QNLI (Question-answering NLI) dataset is a Natural Language Inference dataset automatically derived from the Stanford Question Answering Dataset v1.1 (SQuAD). SQuAD v1.1 consists of question-paragraph pairs, where one of the sentences in the paragraph (drawn from Wikipedia) contains the answer to the corresponding question (written by an annotator). The dataset was converted into sentence pair classification by forming a pair between each question and each sentence in the corresponding context, and filtering out pairs with low lexical overlap between the question and the context sentence. The task is to determine whether the context sentence contains the answer to the question. This modified version of the original task removes the requirement that the model select the exact answer, but also removes the simplifying assumptions that the answer is always present in the input and that lexical overlap is a reliable cue. The QNLI dataset is part of GLUE benchmark. Source: https://arxiv.org/pdf/1804.07461.pdf

The Stanford Question Answering Dataset (SQuAD) is a collection of question-answer pairs derived from Wikipedia articles. In SQuAD, the correct answers of questions can be any sequence of tokens in the given text. Because the questions and answers are produced by humans through crowdsourcing, it is more diverse than some other question-answering datasets. SQuAD 1.1 contains 107,785 question-answer pairs on 536 articles. SQuAD2.0 (open-domain SQuAD, SQuAD-Open), the latest version, combines the 100,000 questions in SQuAD1.1 with over 50,000 un-answerable questions written adversarially by crowdworkers in forms that are similar to the answerable ones. Source: Deep Learning Based Text Classification: A Comprehensive Review Image Source: https://rajpurkar.github.io/SQuAD-explorer/explore/v2.0/dev/Prime_number.html

A new multilingual language model benchmark that is composed of 40+ languages spanning several scripts and linguistic families containing round 40 billion characters and aimed to accelerate the research of multilingual modeling. Source: Wiki-40B: Multilingual Language Model Dataset

The CropAndWeed dataset is focused on the fine-grained identification of 74 relevant crop and weed species with a strong emphasis on data variability. Annotations of labeled bounding boxes, semantic masks and stem positions are provided for about 112k instances in more than 8k high-resolution images of both real-world agricultural sites and specifically cultivated outdoor plots of rare weed types. Additionally, each sample is enriched with meta-annotations regarding environmental conditions.

Europarl-ASR (EN) is a 1300-hour English-language speech and text corpus of parliamentary debates for (streaming) Automatic Speech Recognition training and benchmarking, speech data filtering and speech data verbatimization, based on European Parliament speeches and their official transcripts (1996-2020). Includes dev-test sets for streaming ASR benchmarking, made up of 18 hours of manually revised speeches. The availability of manual non-verbatim and verbatim transcripts for dev-test speeches makes this corpus also useful for the assessment of automatic filtering and verbatimization techniques. The corpus is released under an open licence at https://www.mllp.upv.es/europarl-asr/ Europarl-ASR CONTENTS: [Speech data] 1300 hours of English-language annotated speech data, 3 full sets of timed transcriptions (official non-verbatim, automatically noise-filtered, automatically verbatimized), 18 hours of speech data with both manually revised verbatim transcriptions and official non-verbatim transcriptions, split in 2 independent validation- evaluation partitions for 2 realistic ASR tasks (with vs. without previous knowledge of the speaker); [Text data] 70 million tokens of English-language text data; [Pretrained language models] the Europarl-ASR English-language n-gram language model and vocabulary.

FluidLab is a simulation environment with a diverse set of manipulation tasks involving complex fluid dynamics. These tasks address interactions between solid and fluid as well as among multiple fluids. Source: FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation Image Source: https://arxiv.org/pdf/2303.02346v1.pdf

The Apron Dataset focuses on training and evaluating classification and detection models for airport-apron logistics. In addition to bounding boxes and object categories the dataset is enriched with meta parameters to quantify the models’ robustness against environmental influences.

COCO-N Medium introduces a stochastic benchmark that simulates common real-world scenarios with noticeable label inaccuracies in the COCO dataset. This benchmark combines class and spatial noises to create a challenging yet realistic evaluation framework for instance segmentation models. It mimics datasets manually annotated by crowd workers, where a moderate level of label noise is expected. By incorporating both class and spatial inaccuracies, COCO-N Medium allows researchers to assess their models' basic robustness to label noise, providing insights into performance in typical real-world applications where perfect annotations are rare. This medium-level benchmark serves as a crucial middle ground, offering a more rigorous test than minimally noisy datasets while remaining within the bounds of commonly encountered data quality issues. COCO-N Medium enables a nuanced evaluation of model performance under realistic conditions, helping identify areas for improvement in handling noisy labels and guiding the development of more robust instance segmentation algorithms.

CSTS: Correlation Structures in Time Series CSTS is a comprehensive synthetic benchmarking dataset designed specifically for evaluating correlation structure discovery in time series data. The dataset systematically models known correlation structures between time series variables and enables rigorous assessment of clustering algorithms and validation methods. ## Dataset Characteristics: - 23 distinct correlation structures between three time series variables (iob, cob, ig) - Correlation relationships in three categories: strong positive ([0.7,1]), negligible ([-0.2,0.2]), and strong negative ([-1,-0.7]) - Stationary segments with regime-switching correlation structures, no temporal dependencies (autocorrelation, trends, seasonality) - Segments varying in length from 15 minutes to 10 hours (900-36000 observations) - Data variants include normal distributions, non-normal distributions (similar to insulin, carbohydrates, and glucose data), downsampled versions, and three sparsity levels (complete, partial - missing 30% of observations, sparse - missing 90% of observations) - 60 subjects in total, 30 per data variant, two splits (exploratory/confirmatory) - Ground truth information for segmentation and clustering, and labels for controlled degraded segmentation and clustering ## Motivation: CSTS addresses a critical gap in time series clustering evaluation by providing a structure-first benchmark with well-defined correlation structures rather than arbitrary classification boundaries. This enables researchers to systematically distinguish between correlation structure deterioration and algorithmic limitations, moving clustering analysis from "art" toward science. ## Use Cases: - Evaluating time series clustering algorithms' ability to detect correlation structures under varying data conditions - Assessing clustering validation methods under varying data conditions - Investigating how preprocessing techniques affect correlation structure discovery - Establishing performance thresholds for high-quality clustering results Source: CSTS: A Benchmark for the Discovery of Correlation Structures in Time Series Clustering

GenoTEX (Genomics Data Automatic Exploration Benchmark) is a benchmark dataset for the automated analysis of gene expression data to identify disease-associated genes while considering the influence of other biological factors. It provides analysis code and results for solving a wide range of gene-trait association (GTA) analysis problems, encompassing dataset selection, preprocessing, and statistical analysis, in a pipeline that follows computational genomics standards. The benchmark includes expert-curated annotations from bioinformaticians to ensure accuracy and reliability.

PsOCR is a large-scale synthetic dataset for Optical Character Recognition in low-resource Pashto language.

SemanticSugarBeets, a novel and high-quality dataset containing 953 monocular RGB images and 2920 annotations of sugar beets, enables a wide range of learning tasks including object detection, semantic segmentation, instance segmentation and mass estimation for post-harvest and post-storage analysis.