CNN/Daily Mail is a dataset for text summarization. Human generated abstractive summary bullets were generated from news stories in CNN and Daily Mail websites as questions (with one of the entities hidden), and stories as the corresponding passages from which the system is expected to answer the fill-in the-blank question. The authors released the scripts that crawl, extract and generate pairs of passages and questions from these websites. In all, the corpus has 286,817 training pairs, 13,368 validation pairs and 11,487 test pairs, as defined by their scripts. The source documents in the training set have 766 words spanning 29.74 sentences on an average while the summaries consist of 53 words and 3.72 sentences. Source: Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond

The New York Times Annotated Corpus contains over 1.8 million articles written and published by the New York Times between January 1, 1987 and June 19, 2007 with article metadata provided by the New York Times Newsroom, the New York Times Indexing Service and the online production staff at nytimes.com. The corpus includes: - Over 1.8 million articles (excluding wire services articles that appeared during the covered period). - Over 650,000 article summaries written by library scientists. - Over 1,500,000 articles manually tagged by library scientists with tags drawn from a normalized indexing vocabulary of people, organizations, locations and topic descriptors. - Over 275,000 algorithmically-tagged articles that have been hand verified by the online production staff at nytimes.com. As part of the New York Times' indexing procedures, most articles are manually summarized and tagged by a staff of library scientists. This collection contains over 650,000 article-summary pairs which may prove to be useful in the development and evaluation of algorithms for automated document summarization. Also, over 1.5 million documents have at least one tag. Articles are tagged for persons, places, organizations, titles and topics using a controlled vocabulary that is applied consistently across articles. For instance if one article mentions "Bill Clinton" and another refers to "President William Jefferson Clinton", both articles will be tagged with "CLINTON, BILL". Source: https://catalog.ldc.upenn.edu/LDC2008T19

Multi-News, consists of news articles and human-written summaries of these articles from the site newser.com. Each summary is professionally written by editors and includes links to the original articles cited. Source: Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model Image Source: https://arxiv.org/pdf/1906.01749.pdf

CORNELL NEWSROOM is a large dataset for training and evaluating summarization systems. It contains 1.3 million articles and summaries written by authors and editors in the newsrooms of 38 major publications. The summaries are obtained from search and social metadata between 1998 and 2017 and use a variety of summarization strategies combining extraction and abstraction. Source: CORNELL NEWSROOM

WikiSum is a dataset based on English Wikipedia and suitable for a task of multi-document abstractive summarization. In each instance, the input is comprised of a Wikipedia topic (title of article) and a collection of non-Wikipedia reference documents, and the target is the Wikipedia article text. The dataset is restricted to the articles with at least one crawlable citation. The official split divides the articles roughly into 80/10/10 for train/development/test subsets, resulting in 1865750, 233252, and 232998 examples respectively. Source: Generating Wikipedia by Summarizing Long Sequences Image Source: https://arxiv.org/pdf/1801.10198.pdf

WikiLingua includes ~770k article and summary pairs in 18 languages from WikiHow. Gold-standard article-summary alignments across languages are extracted by aligning the images that are used to describe each how-to step in an article. Source: https://github.com/esdurmus/Wikilingua Image Source: Ladhak et al

BookSum is a collection of datasets for long-form narrative summarization. This dataset covers source documents from the literature domain, such as novels, plays and stories, and includes highly abstractive, human written summaries on three levels of granularity of increasing difficulty: paragraph-, chapter-, and book-level. The domain and structure of this dataset poses a unique set of challenges for summarization systems, which include: processing very long documents, non-trivial causal and temporal dependencies, and rich discourse structures. BookSum contains summaries for 142,753 paragraphs, 12,293 chapters and 436 books.

Arxiv HEP-TH (high energy physics theory) citation graph is from the e-print arXiv and covers all the citations within a dataset of 27,770 papers with 352,807 edges. If a paper i cites paper j, the graph contains a directed edge from i to j. If a paper cites, or is cited by, a paper outside the dataset, the graph does not contain any information about this. The data covers papers in the period from January 1993 to April 2003 (124 months). Source: https://snap.stanford.edu/data/cit-HepTh.html

The WCEP dataset for multi-document summarization (MDS) consists of short, human-written summaries about news events, obtained from the Wikipedia Current Events Portal (WCEP), each paired with a cluster of news articles associated with an event. These articles consist of sources cited by editors on WCEP, and are extended with articles automatically obtained from the Common Crawl News dataset. Source: WCEP

To study the task of email subject line generation: automatically generating an email subject line from the email body. Source: This Email Could Save Your Life: Introducing the Task of Email Subject Line Generation

Large-scale manually-annotated corpus for 1,000 scientific papers (on computational linguistics) for automatic summarization. Summaries for each paper are constructed from the papers that cite that paper and from that paper's abstract. Source: ScisummNet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networks

This is a dataset for evaluating summarisation methods for research papers. Source: A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents

5,519 query-based summaries, each associated with an average of 6 input documents selected from an index of 355M documents from Common Crawl. Source: AQuaMuSe: Automatically Generating Datasets for Query-Based Multi-Document Summarization

Multi-XScience is a large-scale dataset for multi-document summarization of scientific articles. It has 30,369, 5,066 and 5,093 samples for the train, validation and test split respectively. The average document length is 778.08 words and the average summary length is 116.44 words. Source: https://github.com/yaolu/Multi-XScience

CQASUMM is a dataset for CQA (Community Question Answering) summarization, constructed from the 4.4 million Yahoo! Answers L6 dataset. The dataset contains ~300k annotated samples. Source: CQASUMM

EUR-Lex-Sum is a dataset for cross-lingual summarization. It is based on manually curated document summaries of legal acts from the European Union law platform. Documents and their respective summaries exist as crosslingual paragraph-aligned data in several of the 24 official European languages, enabling access to various cross-lingual and lower-resourced summarization setups. The dataset contains up to 1,500 document/summary pairs per language, including a subset of 375 cross-lingually aligned legal acts with texts available in all 24 languages. Source: EUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domain Image Source: https://arxiv.org/pdf/2210.13448v1.pdf

OPOSUM is a dataset for the training and evaluation of Opinion Summarization models which contains Amazon reviews from six product domains: Laptop Bags, Bluetooth Headsets, Boots, Keyboards, Televisions, and Vacuums. The six training collections were created by downsampling from the Amazon Product Dataset introduced in McAuley et al. (2015) and contain reviews and their respective ratings. A subset of the dataset has been manually annotated, specifically, for each domain, 10 different products were uniformly sampled (across ratings) with 10 reviews each, amounting to a total of 600 reviews, to be used only for development (300) and testing (300). Source: Summarizing Opinions: Aspect Extraction Meets Sentiment Prediction and They Are Both Weakly Supervised

The TalkSumm dataset contains 1705 automatically-generated summaries of scientific papers from ACL, NAACL, EMNLP, SIGDIAL (2015-2018), and ICML (2017-2018). The dataset is provided as a list of titles and URLs and the corresponding summaries. Source: GitHub

Maternal and Infant (MATINF) Dataset is a large-scale dataset jointly labeled for classification, question answering and summarization in the domain of maternity and baby caring in Chinese. An entry in the dataset includes four fields: question (Q), description (D), class (C) and answer (A). Nearly two million question-answer pairs are collected with fine-grained human-labeled classes from a large Chinese maternity and baby caring QA site. Authors conduct both automatic and manual data cleansing and remove: (1) classes with insufficient samples; (2) entries in which the length of the description filed is less than the length of the question field; (3) data with any field longer than 256 characters; (4) human-spotted ill-formed data. After the data cleansing, MATINF is constructed with the remaining 1.07 million entries Source: MATINF: A Jointly Labeled Large-Scale Dataset for Classification, Question Answering and Summarization

FacetSum is a faceted summarization dataset for scientific documents. FacetSum has been built on Emerald journal articles, covering a diverse range of domains. Different from traditional document-summary pairs, FacetSum provides multiple summaries, each targeted at specific sections of a long document, including the purpose, method, findings, and value.

Wikipedia Generation is a dataset for article generation from Wikipedia from references at the end of Wikipedia page and the top 10 search results for the Wikipedia topic.

Email Thread Summarization (EmailSum) is a dataset which contains human-annotated short (<30 words) and long (<100 words) summaries of 2,549 email threads (each containing 3 to 10 emails) over a wide variety of topics. It was developed to spur research in thread summarization. Source: EMAILSUM: Abstractive Email Thread Summarization Image source: EMAILSUM: Abstractive Email Thread Summarization

An open corpus of Scientific Research papers which has a representative sample from across scientific disciplines. This corpus not only includes the full text of the article, but also the metadata of the documents, along with the bibliographic information for each reference. Source: Elsevier OA CC-By Corpus

FINDSum is a large-scale dataset for long text and multi-table summarization. It is built on 21,125 annual reports from 3,794 companies and has two subsets for summarizing each company’s results of operations and liquidity. Source: Long Text and Multi-Table Summarization: Dataset and Method Image Source: https://aclanthology.org/2022.findings-emnlp.145.pdf

GameWikiSum is a domain-specific (video game) dataset for multi-document summarization, which is one hundred times larger than commonly used datasets, and in another domain than news. Input documents consist of long professional video game reviews as well as references of their gameplay sections in Wikipedia pages. Source: https://github.com/Diego999/GameWikiSum

HowSumm is a large-scale query-focused multi-document summarization dataset. It is focused on summarization of various sources to create HowTo guides. It is derived from wikiHow articles. HowSumm is partitioned into HowSumm-Step where target summary is relatively short (avg. 90 words) and HowSumm-Method where target summary is a concatenation of several steps an therefore longer (avg. 500 words). HowSumm-Method and HowSumm-Step contain 11,121 and 84348 instances, respectively. Description from: HowSumm Image source: HowSumm

PMC-SA (PMC Structured Abstracts) is a dataset of academic publications, used for the task of structured summarization. Source: https://arxiv.org/abs/1905.07695

A high-quality large-scale dataset consisting of 49,000+ data samples for the task of Chinese query-based document summarization. Source: QBSUM: a Large-Scale Query-Based Document Summarization Dataset from Real-world Applications

SubSumE Dataset This repository contains the SubSumE dataset for subjective document summarization. See the paper and the talk for details on dataset creation. Also check out our work SuDocu on example-based document summarization. ## Dataset Files Download the dataset from here. The dataset contains : * Simplified text from 48 Wikipedia pages of the states in the US. Additionally, all the sentences in these documents are put together in a single file processed_state_sentences.csv and are assigned a unique sentence id that is used in summary json files. * Intent-based summaries created by human annotators. Each datapoint file in the directory user_summary_jsons contains a json containing summaries of Wikipedia pages of eight states with following keys: * intent : Summarization intent provided to human annotators for generating the summary * summaries: List of summary jsons for eight states assigned to the annotator. Each json in the list contains following keys: * state_name: Name of the state * sentence_ids: Global ids of sentences (wrt processed_state_sentences.csv) present in the summary * sentences: List of sentences present in the summary * use_keywords: Keywords used by the annotator to search the document when creating summaries