Text Summarization

The PubMed dataset consists of 19717 scientific publications from PubMed database pertaining to diabetes classified into one of three classes. The citation network consists of 44338 links. Each publication in the dataset is described by a TF/IDF weighted word vector from a dictionary which consists of 500 unique words. Source: https://linqs.soe.ucsc.edu/data

The Reddit dataset is a graph dataset from Reddit posts made in the month of September, 2014. The node label in this case is the community, or “subreddit”, that a post belongs to. 50 large communities have been sampled to build a post-to-post graph, connecting posts if the same user comments on both. In total this dataset contains 232,965 posts with an average degree of 492. The first 20 days are used for training and the remaining days for testing (with 30% used for validation). For features, off-the-shelf 300-dimensional GloVe CommonCrawl word vectors are used. Source: https://arxiv.org/pdf/1706.02216.pdf Image Source: https://minimaxir.com/2016/05/reddit-graph/

CNN/Daily Mail is a dataset for text summarization. Human generated abstractive summary bullets were generated from news stories in CNN and Daily Mail websites as questions (with one of the entities hidden), and stories as the corresponding passages from which the system is expected to answer the fill-in the-blank question. The authors released the scripts that crawl, extract and generate pairs of passages and questions from these websites. In all, the corpus has 286,817 training pairs, 13,368 validation pairs and 11,487 test pairs, as defined by their scripts. The source documents in the training set have 766 words spanning 29.74 sentences on an average while the summaries consist of 53 words and 3.72 sentences. Source: Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond

The New York Times Annotated Corpus contains over 1.8 million articles written and published by the New York Times between January 1, 1987 and June 19, 2007 with article metadata provided by the New York Times Newsroom, the New York Times Indexing Service and the online production staff at nytimes.com. The corpus includes: - Over 1.8 million articles (excluding wire services articles that appeared during the covered period). - Over 650,000 article summaries written by library scientists. - Over 1,500,000 articles manually tagged by library scientists with tags drawn from a normalized indexing vocabulary of people, organizations, locations and topic descriptors. - Over 275,000 algorithmically-tagged articles that have been hand verified by the online production staff at nytimes.com. As part of the New York Times' indexing procedures, most articles are manually summarized and tagged by a staff of library scientists. This collection contains over 650,000 article-summary pairs which may prove to be useful in the development and evaluation of algorithms for automated document summarization. Also, over 1.5 million documents have at least one tag. Articles are tagged for persons, places, organizations, titles and topics using a controlled vocabulary that is applied consistently across articles. For instance if one article mentions "Bill Clinton" and another refers to "President William Jefferson Clinton", both articles will be tagged with "CLINTON, BILL". Source: https://catalog.ldc.upenn.edu/LDC2008T19

CORD-19 is a free resource of tens of thousands of scholarly articles about COVID-19, SARS-CoV-2, and related coronaviruses for use by the global research community. Source: Allen Institute for AI

MTEB is a benchmark that spans 8 embedding tasks covering a total of 56 datasets and 112 languages. The 8 task types are Bitext mining, Classification, Clustering, Pair Classification, Reranking, Retrieval, Semantic Textual Similarity and Summarisation. The 56 datasets contain varying text lengths and they are grouped into three categories: Sentence to sentence, Paragraph to paragraph, and Sentence to paragraph. Check the latest leaderboards at HuggingFace.

A large corpus of 81.1M English-language academic papers spanning many academic disciplines. Rich metadata, paper abstracts, resolved bibliographic references, as well as structured full text for 8.1M open access papers. Full text annotated with automatically-detected inline mentions of citations, figures, and tables, each linked to their corresponding paper objects. Aggregated papers from hundreds of academic publishers and digital archives into a unified source, and create the largest publicly-available collection of machine-readable academic text to date. Source: Allen Institute for AI

A new dataset with abstractive dialogue summaries. Source: SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization

WikiHow is a dataset of more than 230,000 article and summary pairs extracted and constructed from an online knowledge base written by different human authors. The articles span a wide range of topics and represent high diversity styles. Source: WikiHow: A Large Scale Text Summarization Dataset Image Source: WikiHow: A Large Scale Text Summarization Dataset

Multi-News, consists of news articles and human-written summaries of these articles from the site newser.com. Each summary is professionally written by editors and includes links to the original articles cited. Source: Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model Image Source: https://arxiv.org/pdf/1906.01749.pdf

CORNELL NEWSROOM is a large dataset for training and evaluating summarization systems. It contains 1.3 million articles and summaries written by authors and editors in the newsrooms of 38 major publications. The summaries are obtained from search and social metadata between 1998 and 2017 and use a variety of summarization strategies combining extraction and abstraction. Source: CORNELL NEWSROOM

KP20k is a large-scale scholarly articles dataset with 528K articles for training, 20K articles for validation and 20K articles for testing. Source: Keyphrase Prediction With Pre-trained Language Model Image Source: https://arxiv.org/pdf/1704.06879.pdf

The Yelp Dataset is a valuable resource for academic research, teaching, and learning. It provides a rich collection of real-world data related to businesses, reviews, and user interactions. Here are the key details about the Yelp Dataset: Reviews: A whopping 6,990,280 reviews from users. Businesses: Information on 150,346 businesses. Pictures: A collection of 200,100 pictures. Metropolitan Areas: Data from 11 metropolitan areas. Tips: Over 908,915 tips provided by 1,987,897 users. Business Attributes: Details like hours, parking availability, and ambiance for more than 1.2 million businesses. Aggregated Check-ins: Historical check-in data for each of the 131,930 businesses.

The How2 dataset contains 13,500 videos, or 300 hours of speech, and is split into 185,187 training, 2022 development (dev), and 2361 test utterances. It has subtitles in English and crowdsourced Portuguese translations. Source: exploring multiview correlations in open-domain videos

GovReport is a dataset for long document summarization, with significantly longer documents and summaries. It consists of reports written by government research agencies including Congressional Research Service and U.S. Government Accountability Office. Compared with other long document summarization datasets, government report dataset has longer summaries and documents and requires reading in more context to cover salient words to be summarized.

QMSum is a new human-annotated benchmark for query-based multi-domain meeting summarisation task, which consists of 1,808 query-summary pairs over 232 meetings in multiple domains.

XL-Sum is a comprehensive and diverse dataset for abstractive summarization comprising 1 million professionally annotated article-summary pairs from BBC, extracted using a set of carefully designed heuristics. The dataset covers 44 languages ranging from low to high-resource, for many of which no public dataset is currently available. XL-Sum is highly abstractive, concise, and of high quality, as indicated by human and intrinsic evaluation.

Sentence Compression is a dataset where the syntactic trees of the compressions are subtrees of their uncompressed counterparts, and hence where supervised systems which require a structural alignment between the input and output can be successfully trained. Source: Overcoming the Lack of Parallel Data in Sentence Compression

DialogSum is a large-scale dialogue summarization dataset, consisting of 13,460 dialogues with corresponding manually labeled summaries and topics. This work is accepted by ACL findings 2021. You may find the paper here: https://arxiv.org/pdf/2105.06762.pdf. If you want to use our dataset, please cite our paper. #### Dialogue Data We collect dialogue data for DialogSum from three public dialogue corpora, namely Dailydialog (Li et al., 2017), DREAM (Sun et al., 2019) and MuTual (Cui et al., 2019), as well as an English speaking practice website. These datasets contain face-to-face spoken dialogues that cover a wide range of daily-life topics, including schooling, work, medication, shopping, leisure, travel. Most conversations take place between friends, colleagues, and between service providers and customers. Compared with previous datasets, dialogues from DialogSum have distinct characteristics: * Under rich real-life scenarios, including more diverse task-oriented scenarios; * Have clear communication patterns and intents, which is valuable to serve as summarization sources; * Have a reasonable length, which comforts the purpose of automatic summarization. #### Summaries We ask annotators to summarize each dialogue based on the following criteria: * Convey the most salient information; * Be brief; * Preserve important named entities within the conversation; * Be written from an observer perspective; * Be written in formal language. #### Topics In addition to summaries, we also ask annotators to write a short topic for each dialogue, which can be potentially useful for future work, e.g. generating summaries by leveraging topic information. Image source: https://arxiv.org/pdf/2105.06762.pdf

SummScreen is a dataset for abstractive screenplay summarization. It consists of pairs of TV series transcripts and human-written recaps. This dataset provides a challenging testbed for abstractive summarization for several reasons: - Plot details are often expressed indirectly in character dialogues and may be scattered across the entirety of the transcript. - These details must be found and integrated to form the succinct plot descriptions in the recaps. - TV scripts contain content that does not directly pertain to the central plot but rather serves to develop characters or provide comic relief. This information is rarely contained in recaps.

LCSTS is a large corpus of Chinese short text summarization dataset constructed from the Chinese microblogging website Sina Weibo, which is released to the public. This corpus consists of over 2 million real Chinese short texts with short summaries given by the author of each text. The authors also manually tagged the relevance of 10,666 short summaries with their corresponding short texts 10,666 short summaries with their corresponding short texts. Source: LCSTS: A Large Scale Chinese Short Text Summarization Dataset

WikiSum is a dataset based on English Wikipedia and suitable for a task of multi-document abstractive summarization. In each instance, the input is comprised of a Wikipedia topic (title of article) and a collection of non-Wikipedia reference documents, and the target is the Wikipedia article text. The dataset is restricted to the articles with at least one crawlable citation. The official split divides the articles roughly into 80/10/10 for train/development/test subsets, resulting in 1865750, 233252, and 232998 examples respectively. Source: Generating Wikipedia by Summarizing Long Sequences Image Source: https://arxiv.org/pdf/1801.10198.pdf

WikiLingua includes ~770k article and summary pairs in 18 languages from WikiHow. Gold-standard article-summary alignments across languages are extracted by aligning the images that are used to describe each how-to step in an article. Source: https://github.com/esdurmus/Wikilingua Image Source: Ladhak et al

Consists of 1.3 million records of U.S. patent documents along with human written abstractive summaries. Source: BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization

BillSum is the first dataset for summarization of US Congressional and California state bills. The BillSum dataset consists of three parts: US training bills, US test bills and California test bills. The US bills were collected from the Govinfo service provided by the United States Government Publishing Office (GPO). The corpus consists of bills from the 103rd-115th (1993-2018) sessions of Congress. The data was split into 18,949 train bills and 3,269 test bills. For California, bills from the 2015-2016 session were scraped directly from the legislature’s website; the summaries were written by their Legislative Counsel. The BillSum corpus focuses on mid-length legislation from 5,000 to 20,000 character in length. The authors chose to measure the text length in characters, instead of words or sentences, because the texts have complex structure that makes it difficult to consistently measure words. The range was chosen because on one side, short bills introduce minor changes and do not require summaries. While the CRS produces summaries for them, they often contain most of the text of the bill. On the other side, very long legislation is often composed of several large sections. Source: BillSum: A Corpus for Automatic Summarization of US Legislation

Reddit TIFU dataset is a newly collected Reddit dataset, where TIFU denotes the name of /r/tifu subbreddit. There are 122,933 text-summary pairs in total. Source: https://github.com/ctr4si/MMN

A large-scale MultiLingual SUMmarization dataset. Obtained from online newspapers, it contains 1.5M+ article/summary pairs in five different languages -- namely, French, German, Spanish, Russian, Turkish. Together with English newspapers from the popular CNN/Daily mail dataset, the collected data form a large scale multilingual dataset which can enable new research directions for the text summarization community. Source: MLSUM: The Multilingual Summarization Corpus

** SCROLLS** (Standardized CompaRison Over Long Language Sequences) is an NLP benchmark consisting of a suite of tasks that require reasoning over long texts. SCROLLS contains summarization, question answering, and natural language inference tasks, covering multiple domains, including literature, science, business, and entertainment. The dataset is made available in a unified text-to-text format and host a live leaderboard to facilitate research on model architecture and pretraining methods. The SCROLLS benchmark contains the datasets GovReport, SummScreenFD, QMSum, QASPER, NarrativeQA, QuALITY and ContractNLI.

BookSum is a collection of datasets for long-form narrative summarization. This dataset covers source documents from the literature domain, such as novels, plays and stories, and includes highly abstractive, human written summaries on three levels of granularity of increasing difficulty: paragraph-, chapter-, and book-level. The domain and structure of this dataset poses a unique set of challenges for summarization systems, which include: processing very long documents, non-trivial causal and temporal dependencies, and rich discourse structures. BookSum contains summaries for 142,753 paragraphs, 12,293 chapters and 436 books.

Arxiv HEP-TH (high energy physics theory) citation graph is from the e-print arXiv and covers all the citations within a dataset of 27,770 papers with 352,807 edges. If a paper i cites paper j, the graph contains a directed edge from i to j. If a paper cites, or is cited by, a paper outside the dataset, the graph does not contain any information about this. The data covers papers in the period from January 1993 to April 2003 (124 months). Source: https://snap.stanford.edu/data/cit-HepTh.html

xP3 is a multilingual dataset for multitask prompted finetuning. It is a composite of supervised datasets in 46 languages with English and machine-translated prompts. Source: Crosslingual Generalization through Multitask Finetuning Image Source: https://arxiv.org/pdf/2211.01786v1.pdf

MeQSum is a dataset for medical question summarization. It contains 1,000 summarized consumer health questions. Source: https://www.aclweb.org/anthology/P19-1215.pdf Image Source: https://www.aclweb.org/anthology/P19-1215.pdf

The Extreme Summarization (XSum) dataset is a dataset for evaluation of abstractive single-document summarization systems. The goal is to create a short, one-sentence new summary answering the question “What is the article about?”. The dataset consists of 226,711 news articles accompanied with a one-sentence summary. The articles are collected from BBC articles (2010 to 2017) and cover a wide variety of domains (e.g., News, Politics, Sports, Weather, Business, Technology, Science, Health, Family, Education, Entertainment and Arts). The official random split contains 204,045 (90%), 11,332 (5%) and 11,334 (5) documents in training, validation and test sets, respectively. Source: https://arxiv.org/pdf/1808.08745.pdf Image Source: https://arxiv.org/pdf/1808.08745.pdf

The WCEP dataset for multi-document summarization (MDS) consists of short, human-written summaries about news events, obtained from the Wikipedia Current Events Portal (WCEP), each paired with a cluster of news articles associated with an event. These articles consist of sources cited by editors on WCEP, and are extended with articles automatically obtained from the Common Crawl News dataset. Source: WCEP

license: apache-2.0 tags: human-feedback size_categories: 100K<n<1M pretty_name: OpenAssistant Conversations # OpenAssistant Conversations Dataset (OASST1) ## Dataset Description - Homepage: https://www.open-assistant.io/ - Repository: https://github.com/LAION-AI/Open-Assistant - Paper: https://arxiv.org/abs/2304.07327 ### Dataset Summary In an effort to democratize research on large-scale alignment, we release OpenAssistant Conversations (OASST1), a human-generated, human-annotated assistant-style conversation corpus consisting of 161,443 messages in 35 different languages, annotated with 461,292 quality ratings, resulting in over 10,000 fully annotated conversation trees. The corpus is a product of a worldwide crowd-sourcing effort involving over 13,500 volunteers. Please refer to our paper for further details. ### Dataset Structure This dataset contains message trees. Each message tree has an initial prompt message as the root node, which can have multiple child messages as replies, and these child messages can have multiple replies. All messages have a role property: this can either be "assistant" or "prompter". The roles in conversation threads from prompt to leaf node strictly alternate between "prompter" and "assistant". This version of the dataset contains data collected on the open-assistant.io website until April 12 2023. ### JSON Example: Message For readability, the following JSON examples are shown formatted with indentation on multiple lines. Objects are stored without indentation (on single lines) in the actual jsonl files. json { "message_id": "218440fd-5317-4355-91dc-d001416df62b", "parent_id": "13592dfb-a6f9-4748-a92c-32b34e239bb4", "user_id": "8e95461f-5e94-4d8b-a2fb-d4717ce973e4", "text": "It was the winter of 2035, and artificial intelligence (..)", "role": "assistant", "lang": "en", "review_count": 3, "review_result": true, "deleted": false, "rank": 0, "synthetic": true, "model_name": "oasst-sft-0_3000,max_new_tokens=400 (..)", "labels": { "spam": { "value": 0.0, "count": 3 }, "lang_mismatch": { "value": 0.0, "count": 3 }, "pii": { "value": 0.0, "count": 3 }, "not_appropriate": { "value": 0.0, "count": 3 }, "hate_speech": { "value": 0.0, "count": 3 }, "sexual_content": { "value": 0.0, "count": 3 }, "quality": { "value": 0.416, "count": 3 }, "toxicity": { "value": 0.16, "count": 3 }, "humor": { "value": 0.0, "count": 3 }, "creativity": { "value": 0.33, "count": 3 }, "violence": { "value": 0.16, "count": 3 } } } ### JSON Example: Conversation Tree For readability, only a subset of the message properties is shown here. json { "message_tree_id": "14fbb664-a620-45ce-bee4-7c519b16a793", "tree_state": "ready_for_export", "prompt": { "message_id": "14fbb664-a620-45ce-bee4-7c519b16a793", "text": "Why can't we divide by 0? (..)", "role": "prompter", "lang": "en", "replies": [ { "message_id": "894d30b6-56b4-4605-a504-89dd15d4d1c8", "text": "The reason we cannot divide by zero is because (..)", "role": "assistant", "lang": "en", "replies": [ // ... ] }, { "message_id": "84d0913b-0fd9-4508-8ef5-205626a7039d", "text": "The reason that the result of a division by zero is (..)", "role": "assistant", "lang": "en", "replies": [ { "message_id": "3352725e-f424-4e3b-a627-b6db831bdbaa", "text": "Math is confusing. Like those weird Irrational (..)", "role": "prompter", "lang": "en", "replies": [ { "message_id": "f46207ca-3149-46e9-a466-9163d4ce499c", "text": "Irrational numbers are simply numbers (..)", "role": "assistant", "lang": "en", "replies": [] }, // ... ] } ] } ] } } Please refer to oasst-data for details about the data structure and Python code to read and write jsonl files containing oasst data objects. If you would like to explore the dataset yourself you can find a getting-started notebook in the notebooks/openassistant-oasst1 folder of the LAION-AI/Open-Assistant github repository. ## Main Dataset Files Conversation data is provided either as nested messages in trees (extension .trees.jsonl.gz) or as a flat list (table) of messages (extension .messages.jsonl.gz). ### Ready For Export Trees 2023-04-12_oasst_ready.trees.jsonl.gz 10,364 trees with 88,838 total messages 2023-04-12_oasst_ready.messages.jsonl.gz 88,838 messages Trees in ready_for_export state without spam and deleted messages including message labels. The oasst_ready-trees file usually is sufficient for supervised fine-tuning (SFT) & reward model (RM) training. ### All Trees 2023-04-12_oasst_all.trees.jsonl.gz 66,497 trees with 161,443 total messages 2023-04-12_oasst_all.messages.jsonl.gz 161,443 messages All trees, including those in states prompt_lottery_waiting (trees that consist of only one message, namely the initial prompt), aborted_low_grade (trees that stopped growing because the messages had low quality), and halted_by_moderator. ### Supplemental Exports: Spam & Prompts 2023-04-12_oasst_spam.messages.jsonl.gz These are messages which were deleted or have a negative review result ("review_result": false). Besides low quality, a frequent reason for message deletion is a wrong language tag. 2023-04-12_oasst_prompts.messages.jsonl.gz These are all the kept initial prompt messages with positive review result (no spam) of trees in ready_for_export or prompt_lottery_waiting state. ### Using the Huggingface Datasets While HF datasets is ideal for tabular datasets, it is not a natural fit for nested data structures like the OpenAssistant conversation trees. Nevertheless, we make all messages which can also be found in the file 2023-04-12_oasst_ready.trees.jsonl.gz available in parquet as train/validation splits. These are directly loadable by Huggingface Datasets. To load the oasst1 train & validation splits use: python from datasets import load_dataset ds = load_dataset("OpenAssistant/oasst1") train = ds['train'] # len(train)=84437 (95%) val = ds['validation'] # len(val)=4401 (5%) The messages appear in depth-first order of the message trees. Full conversation trees can be reconstructed from the flat messages table by using the parent_id and message_id properties to identify the parent-child relationship of messages. The message_tree_id and tree_state properties (only present in flat messages files) can be used to find all messages of a message tree or to select trees by their state. ### Languages OpenAssistant Conversations incorporates 35 different languages with a distribution of messages as follows: Languages with over 1000 messages - English: 71956 - Spanish: 43061 - Russian: 9089 - German: 5279 - Chinese: 4962 - French: 4251 - Thai: 3042 - Portuguese (Brazil): 2969 - Catalan: 2260 - Korean: 1553 - Ukrainian: 1352 - Italian: 1320 - Japanese: 1018 <details> <summary><b>Languages with under 1000 messages</b></summary> <ul> <li>Vietnamese: 952</li> <li>Basque: 947</li> <li>Polish: 886</li> <li>Hungarian: 811</li> <li>Arabic: 666</li> <li>Dutch: 628</li> <li>Swedish: 512</li> <li>Turkish: 454</li> <li>Finnish: 386</li> <li>Czech: 372</li> <li>Danish: 358</li> <li>Galician: 339</li> <li>Hebrew: 255</li> <li>Romanian: 200</li> <li>Norwegian Bokmål: 133</li> <li>Indonesian: 115</li> <li>Bulgarian: 95</li> <li>Bengali: 82</li> <li>Persian: 72</li> <li>Greek: 66</li> <li>Esperanto: 59</li> <li>Slovak: 19</li> </ul> </details> ## Contact - Discord Open Assistant Discord Server - GitHub: LAION-AI/Open-Assistant - E-Mail: open-assistant@laion.ai

This dataset contains 4,828 full biomedical articles paired with non-technical lay summaries derived from the eLife scientific journal.