The Caltech-UCSD Birds-200-2011 (CUB-200-2011) dataset is the most widely-used dataset for fine-grained visual categorization task. It contains 11,788 images of 200 subcategories belonging to birds, 5,994 for training and 5,794 for testing. Each image has detailed annotations: 1 subcategory label, 15 part locations, 312 binary attributes and 1 bounding box. The textual information comes from Reed et al.. They expand the CUB-200-2011 dataset by collecting fine-grained natural language descriptions. Ten single-sentence descriptions are collected for each image. The natural language descriptions are collected through the Amazon Mechanical Turk (AMT) platform, and are required at least 10 words, without any information of subcategories and actions. Source: Fine-grained Visual-textual Representation Learning Image Source: http://www.vision.caltech.edu/visipedia/CUB-200-2011.html

The IMDb Movie Reviews dataset is a binary sentiment analysis dataset consisting of 50,000 reviews from the Internet Movie Database (IMDb) labeled as positive or negative. The dataset contains an even number of positive and negative reviews. Only highly polarizing reviews are considered. A negative review has a score ≤ 4 out of 10, and a positive review has a score ≥ 7 out of 10. No more than 30 reviews are included per movie. The dataset contains additional unlabeled data. Source: http://nlpprogress.com/english/sentiment_analysis.html Image Source: Maas et al

The MPQA Opinion Corpus contains 535 news articles from a wide variety of news sources manually annotated for opinions and other private states (i.e., beliefs, emotions, sentiments, speculations, etc.). Source: http://www.cs.cornell.edu/home/llee/omsa/omsa.pdf Image Source: https://mpqa.cs.pitt.edu/

For understanding multimodal language used in expressing humor. Source: UR-FUNNY: A Multimodal Language Dataset for Understanding Humor

The Norwegian Review Corpus (NoReC) was created for the purpose of training and evaluating models for document-level sentiment analysis. More than 43,000 full-text reviews have been collected from major Norwegian news sources and cover a range of different domains, including literature, movies, video games, restaurants, music and theater, in addition to product reviews across a range of categories. Each review is labeled with a manually assigned score of 1–6, as provided by the rating of the original author. Source: NoReC: The Norwegian Review Corpus

A large-scale stance detection dataset from comments written by candidates of elections in Switzerland. The dataset consists of German, French and Italian text, allowing for a cross-lingual evaluation of stance detection. It contains 67 000 comments on more than 150 political issues (targets). Source: X-Stance: A Multilingual Multi-Target Dataset for Stance Detection

The Cornell eRulemaking Corpus – CDCP is an argument mining corpus annotated with argumentative structure information capturing the evaluability of arguments. The corpus consists of 731 user comments on Consumer Debt Collection Practices (CDCP) rule by the Consumer Financial Protection Bureau (CFPB); the resulting dataset contains 4931 elementary unit and 1221 support relation annotations. It is a resource for building argument mining systems that can not only extract arguments from unstructured text, but also identify what additional information is necessary for readers to understand and evaluate a given argument. Immediate applications include providing real-time feedback to commenters, specifying which types of support for which propositions can be added to construct better-formed arguments.

These are 10 synthetic genomics datasets generated with NEAT v3 (based on TP53 gene of Homo Sapiens) for the use case of benchmarking somatic variant callers. To find more about our generating framework please visit synth4bench GitHub repository. The datasets explore intrinsic NGS data parameters for the use case of observing their effect on tumor-only somatic variant calling algorithms. From the 10 datasets, there are 5 of them with different coverage (while keeping all other parameters fixed) and 5 with varying read length. The reads in all datasets are paired-end .

The UKP Argument Annotated Essays corpus consists of argument annotated persuasive essays including annotations of argument components and argumentative relations. Source: https://www.informatik.tu-darmstadt.de/ukp/research_6/data/argumentation_mining_1/argument_annotated_essays/index.en.jsp Image Source: https://www.aclweb.org/anthology/C14-1142.pdf

DebateSum consists of 187328 debate documents, arguments (also can be thought of as abstractive summaries, or queries), word-level extractive summaries, citations, and associated metadata organized by topic-year. This data is ready for analysis by NLP systems. Source: https://github.com/Hellisotherpeople/DebateSum

Data set covering a set of debatable topics, where for each topic and stance, a set of triplets of the form <argument, KP, label> is provided. The data set is based on the ArgKP data set, which contains arguments contributed by the crowd on 28 debatable topics, split by their stance towards the topic, and KPs written by an expert for those topics. Crowd annotations were collected to determine whether a KP represents an argument, i.e., is a match for an argument. The arguments in ArgKP are a subset of the [IBM-ArgQ-Rank-30kArgs data set](The arguments in ArgKP are a subset of the IBM-ArgQ-Rank-30kArgs data set). For a test set, we extended ArgKP, adding three new debatable topics, that were also not part of IBM-ArgQ-Rank-30kArgs. The test set was collected specifically for KPA-2021, and was carefully designed to be similar in various aspects to the training data 2 . For each topic, crowd sourced arguments were collected, expert KPs generated, and match/no match annotations for argument/KP pairs obtained, resulting in a data set compatible with the ArgKP format. Arguments collection strictly adhered to the guidelines, quality measures, and post processing used for the collection of arguments in IBM-ArgQ-Rank-30kArgs, while the generation of expert KPs, collection of match annotations, and final data set creation strictly adhered to the manner in which ArgKP was created. Source: [Overview of the 2021 Key Point Analysis Shared Task](Overview of the 2021 Key Point Analysis Shared Task)

The Few-Shot Object Learning (FewSOL) dataset can be used for object recognition with a few images per object. It contains 336 real-world objects with 9 RGB-D images per object from different views. Object segmentation masks, object poses and object attributes are provided. In addition, synthetic images generated using 330 3D object models are used to augment the dataset. FewSOL dataset can be used to study a set of few-shot object recognition problems such as classification, detection and segmentation, shape reconstruction, pose estimation, keypoint correspondences and attribute recognition. Motivation: If robots can recognize objects from a few exemplar images, it is possible to scale up the number of objects a robot can recognize because collecting a few images per object is a much easier process compared to building a 3D model of an object. In addition, models trained in the meta-learning setting can generalize to new objects without re-training.

Review-Rebuttal (RR) dataset is introduced to facilitate the study of argument pair extraction in the peer review and rebuttal domain.

XED is a multilingual fine-grained emotion dataset. The dataset consists of human-annotated Finnish (25k) and English sentences (30k), as well as projected annotations for 30 additional languages, providing new resources for many low-resource languages. Source: XED: A Multilingual Dataset for Sentiment Analysis and Emotion Detection

Conversational Stance Detection (CSD) is a dataset with annotations of stances and the structures of conversation threads. It consists of 500 conversation threads (including 500 posts and 5376 comments) from six major social media platforms in Hong Kong. Source: Improved Target-specific Stance Detection on Social Media Platforms by Delving into Conversation Threads Image Source: https://arxiv.org/pdf/2211.03061v1.pdf

We introduce a large and comprehensive dataset to facilitate the study of several essential AM tasks in the debating system. In our work, we first review the existing subtasks (claim extraction, stance classification, evidence extraction), and then propose two integrated argument mining tasks: claim extraction with stance classification (CESC) and claim-evidence pair extraction (CEPE).

SkinCon is a skin disease dataset densely annotated by dermatologists. SkinCon includes 3230 images from the Fitzpatrick 17k skin disease (Fitzpatrick Skin Tone) dataset densely labelled with 48 clinical concepts, 22 of which have at least 50 images representing the concept. The concepts used were chosen by two dermatologists considering the clinical descriptor terms used to describe skin lesions. Examples include "plaque", "scale", and "erosion". Source: SkinCon: A skin disease dataset densely annotated by domain experts for fine-grained model debugging and analysis

Youtbean is a dataset created from closed captions of YouTube product review videos. It can be used for aspect extraction and sentiment classification. Source: https://arxiv.org/pdf/1708.02420.pdf

A new synthetic, multi-purpose dataset - called ENRICH - for testing photogrammetric and computer vision algorithms. Compared to existing datasets, ENRICH offers higher resolution images also rendered with different lighting conditions, camera orientation, scales, and field of view. Specifically, ENRICH is composed of three sub-datasets: ENRICH-Aerial, ENRICH-Square, and ENRICH-Statue, each exhibiting different characteristics. The proposed dataset is useful for several photogrammetry and computer vision-related tasks, such as the evaluation of hand-crafted and deep learning-based local features, effects of ground control points (GCPs) configuration on the 3D accuracy, and monocular depth estimation. Each zip file in the root is relative to a specific dataset: - ENRICH-Aerial, is an aerial image block of the city of Launceston, Australia. The acquisition is performed by simulating a typical oblique aerial camera with five views (nadir and four oblique views). - ENRICH-Square, is a ground-level dataset of a square captured by four cameras, each one moving on a different path with different focal length, orientation, and lighting conditions. - ENRICH-Statue, is a ground-level dataset portraying a statue (placed in the center of the ENRICH-Square scene), acquired using four cameras moving on different paths with different focal lengths, orientations, and lighting conditions. Be sure to check the README file in the dataset root for information on folder structure and file contents. Please refer to the related paper (https://doi.org/10.1016/j.isprsjprs.2023.03.002) for information about the generation method and the purpose of ENRICH.

FinnSentiment introduces a 27,000 sentence dataset (in Finnish) annotated independently with sentiment polarity by three native annotators. Source: FinnSentiment -- A Finnish Social Media Corpus for Sentiment Polarity Annotation

Morph Call is a suite of 46 probing tasks for four Indo-European languages that fall under different morphology: Russian, French, English, and German. The tasks are designed to explore the morphosyntactic content of multilingual transformers which is a less studied aspect at the moment. The tasks are divided into four groups: * Morphosyntactic Features: probe the encoder for the occurrence of the morphosyntactic properties. * Masked Token: analogous to Morphosyntactic Features with the exception that the target word is replaced with a tokenizer-specific mask token. * Morphosyntactic Values: is a group of k-way classification tasks for each feature where k is the number of values that the feature can take. * Perturbations: tasks test the encoder sensitivity to syntactic and inflectional sentence perturbations. ## Probing Methods * Supervised probing involves training a Logistic Regression classifier to predict a property. The performance is used as a proxy to evaluate the model knowledge. * Neuron-level Analysis [Durrani et al., 2020] allows retrieving a group of individual neurons that are most relevant to predict a linguistic property. * Contextual Correlation Analysis [Wu et al., 2020] is a representation-level similarity measure that allows identifying pairs of layers of similar behavior. ## Usage We provide an example of the experiment on Masked Token task (Case, German). bash me@my-laptop:~$ python3 probe.py --help INFO: Showing help with the command 'probe.py -- --help'. NAME probe.py - configure the experiment AND perform probing SYNOPSIS probe.py <flags> DESCRIPTION configure the experiment AND perform probing FLAGS --results_path=RESULTS_PATH Type: Optional[str] Default: None path to a folder to store the probing results and the model intermediate activations --model_architecture=MODEL_ARCHITECTURE Type: typ... Default: 'bert multilingual' --model_is_finetuned=MODEL_IS_FINETUNED Type: bool Default: False if to perform the experiment on the fine-tuned model --model_finetuned_path=MODEL_FINETUNED_PATH Type: Optional[str] Default: None (only if model_is_finetuned is True) path to store the fine-tuned model --model_finetuned_config_google_url=MODEL_FINETUNED_CONFIG_GOOGLE_URL Type: Optional[] Default: None (only if model_is_finetuned is True) the url of the fine-tuned model config if to be downloaded --model_finetuned_model_google_url=MODEL_FINETUNED_MODEL_GOOGLE_URL Type: Optional[] Default: None (only if model_is_finetuned is True) the url of the fine-tuned model weights if to be downloaded --model_is_random=MODEL_IS_RANDOM Type: bool Default: False if to perform the random initialization of the model --layers_to_probe=LAYERS_TO_PROBE Type: List Default: 'all' (either "all" or list w. possible numbers from 0 to 11) -- model layers to probe. e.g.: [1, 3, 11], or "all" --train_n_sentences=TRAIN_N_SENTENCES Type: int Default: 1500 number of sentences used to train the probing classifier --test_n_sentences=TEST_N_SENTENCES Type: int Default: 1000 number of sentences used to evaluate the probing classifier --dev_n_sentences=DEV_N_SENTENCES Type: int Default: 0 DEPRECATED

UIT-ViSFD is a Vietnamese Smartphone Feedback Dataset as a new benchmark corpus built based on strict annotation schemes for evaluating aspect-based sentiment analysis, consisting of 11,122 human-annotated comments for mobile e-commerce, which is freely available for research purposes.

Binary labels for Validity and Novelty respectively are given for each Conclusion.

Freely licensed dataset with warrants for 2k authentic arguments from news comments. On this basis, we present a new challenging task, the argument reasoning comprehension task. Given an argument with a claim and a premise, the goal is to choose the correct implicit warrant from two options. Both warrants are plausible and lexically close, but lead to contradicting claims.

An artificial corpus built using grammatical dependencies rules due to the lack of resources for Sign Language. Source: ASLG-PC12

Corpus of controversial news articles extracted from Twitter. Contains news from three different topics: Beef Ban – controversy over the slaughter and sale of beef on religious grounds (1543 articles) is localised to a particular region, mainly Indian subcontinent, while Gun Control – restrictions on carrying, using, or purchasing firearms (6494 articles) and Capital Punishment – use of the death penalty (7905 articles) are topical in various regions around the world. Source: Comparative Document Summarisation via Classification

Corpus for argument mining in legal documents, composed of 40 decisions of the Court of Justice of the European Union on matters of fiscal state aid

This dataset was used in the paper 'Template-based Abstractive Microblog Opinion Summarisation' (to be published at TACL, 2022). The data is structured as follows: each file represents a cluster of tweets which contains the tweet IDs and a summary of the tweets written by journalists. The gold standard summary follows a template structure and depending on its opinion content, it contains a main story, majority opinion (if any) and/or minority opinions (if any).

We introduce OpenDebateEvidence, a comprehensive dataset for argument mining and summarization sourced from the American Competitive Debate community. This dataset includes over 3.5 million documents with rich metadata, making it one of the most extensive collections of debate evidence. OpenDebateEvidence captures the complexity of arguments in high school and college debates, pro- viding valuable resources for training and evaluation. Our extensive experiments demonstrate the efficacy of fine-tuning state-of-the-art large language models for argumentative abstractive summarization across various methods, models, and datasets. By providing this comprehensive resource, we aim to advance com- putational argumentation and support practical applications for debaters, edu- cators, and researchers. OpenDebateEvidence is publicly available to support further research and innovation in computational argumentation. Access it here: https://huggingface.co/datasets/Yusuf5/OpenCaselist

Press Briefing Claim Dataset The dataset contains a total of 53 press briefings from a time span of over four years (2017-2021). While, on average, one press briefing per month is held, the distribution is highly skewed towards recent years. The Press briefings can be categorized into five main thematic categories: Climate, Energy and Mobility, Medicine, Technology and Science. The press briefings are unevenly distributed among the five categories with a major focus on medical press briefings, which also reflects the COVID-19 pandemic. In total, 177 speakers, excluding the hosts, were detected in the dataset. Most of the time the speakers are invited experts. Excluding the top 10 percentile press briefings as outliers, a press briefing had four guests on average. The press briefings consist of 3066 passages, 1719 from guests and journalists and 1294 from hosts. Regarding the sentences, 25040 sentences were collected in total, 5955 from hosts and 18942 from guests.

https://github.com/dialogue-evaluation/RuOpinionNE-2024

Dataset to reproduce results for the paper Detecting CSV file dialects by table uniformity measurement and data type inference, DOI ds240062.

Validity and Novelty are determined in a comparative setting between two conclusions at a time. For Validity and Novelty possible labels are "Conclusion 1 is better", "tie" and "Conclusion 2 is better", for Validity and Novelty respectively.

With the goal of reasoning on the financial textual data, we present a novel dataset for annotating arguments, their components, and relations in the transcripts of earnings conference calls (ECCs).