Binary text classification

ECHR is an English legal judgment prediction dataset of cases from the European Court of Human Rights (ECHR). The dataset contains ~11.5k cases, including the raw text. For each case, the dataset provides a list of facts extracted using regular expressions from the case description. Each case is also mapped to articles of the Convention that were violated (if any). An importance score is also assigned by ECHR. Source: Neural Legal Judgment Prediction in English

TuringBench is a benchmark environment that contains : - Benchmark tasks- Turing Test (i.e., human vs. machine) and Authorship Attribution: (i.e., who is the author of this texts?) - Datasets (Binary and Multi-class settings) - Website with leaderboard The dataset has 20 labels (19 AI text-generators and human). We built this dataset by collecting 10K news articles (mostly Politics) from sources like CNN and only keeping articles with 200-400 words. Next, we used the Titles of these human-written articles to prompt the AI text-generators (ex: GPT-2, GROVER, etc.) to generate 10K articles each. This gives us a sum total of 200K articles and 20 labels. However, since there are two benchmark tasks - Turing Test and Authorship Attribution settings, we have all 20 labels in one dataset for the multi-class setting and only human vs. one AI text-generator, making 19 binary-class datasets.

The TweepFake dataset consists of 25,572 social media messages posted either by bots or humans on Twitter. Each bot imitated a human account and was based on various generative techniques, including Markov Chains, RNN, RNN+Markov, LSTM, and GPT-2.

The Ghostbusters dataset leverages the GPT-3.5-turbo model for generating texts in the domains of creative writing, news, and student essays, providing 2,000 texts in the first two domains and 1,994 in the latter.

It contains 15K triplets of essay problem statements, student-written, and LLM-generated essays. LLMs include ChatGPT, GPT-3.5, and FLAN-T5-XXL. This dataset was created mainly for LLM-generated text detection but can be useful for other tasks as well.

The MAGE dataset provides a large set of generated texts using 27 LLMs from seven different groups: OpenAI GPT, LLaMA, GLM130B, FLAN-T5, OPT, BigScience, and EleutherAI. In total, the dataset contains 432,682 texts, along with two additional sets. The first is an additional test set with texts from unseen domains generated by an unseen model, namely GPT-4. The second set is designed to evaluate the robustness of detectors against paraphrasing attacks. To achieve this, the GPT-3.5-turbo model was employed to paraphrase the sentences from the first set, with all paraphrased texts treated as machine-generated.

MIXSET comprises a total of 3.6k mixtext instances. The dataset features a blend of HWT(human-written text) and MGT(machine-generated text).

DACCORD is a new dataset dedicated to the task of automatically detecting contradictions between sentences in French. The dataset is currently composed of 1034 sentence pairs. It covers the themes of Russia’s invasion of Ukraine in 2022, the Covid-19 pandemic, and the climate crisis.

AGB-DE is a legal NLP corpus for the automated detection of potentially void clauses in German standard form consumer contracts. It consists of 3,764 clauses that have been legally assessed by experts and annotated as potentially void (1) or valid (0). Additionally, each clause is annotated with a topic label.

This is a binary dataset used for Bengali fake review detection in the paper "Bengali Fake Reviews: A Benchmark Dataset and Detection System" accepted in Neurocomputing, a journal published by Elsevier. - Annotated by 4 native Bangla speakers with more than 90% trustworthiness score. - Fleiss' Kappa Score: 0.83 ## Number of Total Data - Fake - 1339 - Non-fake - 7710 ## Class wise statistics of BFRD dataset | Statistics | Fake | Non-fake | |--------------------------|---------|-------------| | Total words | 1,55,789 | 9,27,902 | | Total unique words | 17,739 | 51,200 | | Max Review length | 693 | 1,614 | | Avg number of words | 116.35 | 120.35 | | Avg number of unique words | 84.99 | 88.42 | ## Citation If you use the datasets, please cite the following paper: @article{SHAHARIAR2024127732, title = {Bengali fake reviews: A benchmark dataset and detection system}, journal = {Neurocomputing}, pages = {127732}, year = {2024}, issn = {0925-2312}, doi = {https://doi.org/10.1016/j.neucom.2024.127732}, url = {https://www.sciencedirect.com/science/article/pii/S0925231224005034}, author = {G.M. Shahariar and Md. Tanvir Rouf Shawon and Faisal Muhammad Shah and Mohammad Shafiul Alam and Md. Shahriar Mahbub}, keywords = {Bengali fake reviews detection, Ensemble learning, Transformers, Deep learning, Augmentation, Transliteration}, abstract = {The proliferation of fake reviews on various online platforms has created a major concern for both consumers and businesses. Such reviews can deceive customers and cause damage to the reputation of products or services, making it crucial to identify them. Although the detection of fake reviews has been extensively studied in English language, detecting fake reviews in non-English languages such as Bengali is still a relatively unexplored research area. The novelty of the study unfolds on three fronts: (i) a new publicly available dataset called Bengali Fake Review Detection (BFRD) dataset is introduced, (ii) a unique pipeline has been proposed that translates English words to their corresponding Bengali meaning and also back transliterates Romanized Bengali to Bengali, (iii) a weighted ensemble model that combines four pre-trained transformers model is proposed. The developed dataset consists of 7710 non-fake and 1339 fake food-related reviews collected from social media posts. Rigorous experiments have been conducted to compare multiple deep learning and pre-trained transformer language models and our proposed model to identify the best-performing model. According to the experimental results, the proposed ensemble model attained a weighted F1-score of 0.9843 on a dataset of 13,390 reviews, comprising 1339 actual fake reviews, 5,356 augmented fake reviews, and 6695 reviews randomly selected from the 7710 non-fake instances.} }

We introduce a large semi-automatically generated dataset of ~400,000 descriptive sentences about commonsense knowledge that can be true or false in which negation is present in about 2/3 of the corpus in different forms that we use to evaluate LLMs. Source: ThisIsNotaDataset Image: ThisIsNotaDataset