Papers with Code 아카이브

Text Summarization

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
12
평가표
37
보존된 평가 기록
233

Text Summarization is a natural language processing (NLP) task that involves condensing a lengthy text document into a shorter, more compact version while still retaining the most important information and meaning. The goal is to produce a summary that accurately represents the content of the original text in a concise form.

There are different approaches to text summarization, including extractive methods that identify and extract important sentences or phrases from the text, and abstractive methods that generate new text based on the content of the original text.

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

MATINF

Maternal and Infant (MATINF) Dataset is a large-scale dataset jointly labeled for classification, question answering and summarization in the domain of maternity and baby caring in Chinese. An entry in the dataset includes four fields: question (Q), description (D), class (C) and answer (A). Nearly two million question-answer pairs are collected with fine-grained human-labeled classes from a large Chinese maternity and baby caring QA site. Authors conduct both automatic and manual data cleansing and remove: (1) classes with insufficient samples; (2) entries in which the length of the description filed is less than the length of the question field; (3) data with any field longer than 256 characters; (4) human-spotted ill-formed data. After the data cleansing, MATINF is constructed with the remaining 1.07 million entries Source: MATINF: A Jointly Labeled Large-Scale Dataset for Classification, Question Answering and Summarization

SSN

SSN (short for Semantic Scholar Network) is a scientific papers summarization dataset which contains 141K research papers in different domains and 661K citation relationships. The entire dataset constitutes a large connected citation graph.

WikiHowQA

WikiHowQA is a Community-based Question Answering dataset, which can be used for both answer selection and abstractive summarization tasks. It contains 76,687 questions in the train set, 8,000 in the development set and 22,354 in the test set. Source: WikiHowQA

WITS

This dataset is an extension of MASAC, a multimodal, multi-party, Hindi-English code-mixed dialogue dataset compiled from the popular Indian TV show, ‘Sarabhai v/s Sarabhai’. WITS was created by augmenting MASAC with natural language explanations for each sarcastic dialogue. The dataset consists of the transcribed sarcastic dialogues from 55 episodes of the TV show, along with audio and video multimodal signals. It was designed to facilitate Sarcasm Explanation in Dialogue (SED), a novel task aimed at generating a natural language explanation for a given sarcastic dialogue, that spells out the intended irony. Each data instance in WITS is associated with a corresponding video, audio, and textual transcript where the last utterance is sarcastic in nature. All the final selected explanations contain the following attributes: • Sarcasm source: The speaker in the dialog who is being sarcastic. • Sarcasm target: The person/ thing towards whom the sarcasm is directed. • Action word: Verb/ action used to describe how the sarcasm is taking place. e.g. mocking, insults, taunts, etc. • Description: A description of the scene to help contextualize the sarcasm.

arXiv

For nearly 30 years, ArXiv has served the public and research communities by providing open access to scholarly articles, from the vast branches of physics to the many subdisciplines of computer science to everything in between, including math, statistics, electrical engineering, quantitative biology, and economics. This rich corpus of information offers significant, but sometimes overwhelming depth. In these times of unique global challenges, efficient extraction of insights from data is essential. To help make the arXiv more accessible, we present a free, open pipeline on Kaggle to the machine-readable arXiv dataset: a repository of 1.7 million articles, with relevant features such as article titles, authors, categories, abstracts, full-text PDFs, and more. We hope to empower new use cases that can lead to the exploration of richer machine learning techniques that combine multi-modal features towards applications like trend analysis, paper recommender engines, category prediction, co-citation networks, knowledge graph construction, and semantic search interfaces.

EurekaAlert

This dataset contains around 5000 scholarly articles and their corresponding easy summary from eureka alert blog, the dataset can be used for the combined task of summarization and simplification.

FacetSum

FacetSum is a faceted summarization dataset for scientific documents. FacetSum has been built on Emerald journal articles, covering a diverse range of domains. Different from traditional document-summary pairs, FacetSum provides multiple summaries, each targeted at specific sections of a long document, including the purpose, method, findings, and value.

Gazeta

Gazeta is a dataset for automatic summarization of Russian news. The dataset consists of 63,435 text-summary pairs. To form training, validation, and test datasets, these pairs were sorted by time and the first 52,400 pairs are used as the training dataset, the proceeding 5,265 pairs as the validation dataset, and the remaining 5,770 pairs as the test dataset. Source: https://github.com/IlyaGusev/gazeta

IndoNLG

IndoNLG is a benchmark to measure natural language generation (NLG) progress in three low-resource—yet widely spoken—languages of Indonesia: Indonesian, Javanese, and Sundanese. Altogether, these languages are spoken by more than 100 million native speakers, and hence constitute an important use case of NLG systems today. Concretely, IndoNLG covers six tasks: summarization, question answering, chit-chat, and three different pairs of machine translation (MT) tasks.

MENSA

MENSA: Movie Scene Saliency Dataset ## Dataset Summary The dataset, MENSA (Movie Scene Saliency Dataset) is from the paper "Select and Summarize: Scene Saliency for Movie Script Summarization", and consists of movie scripts and their corresponding summaries. Each scene in the movie script is annotated with scene saliency labels. The training set contains silver labels, which are automatically generated, while the validation and test sets contain human-annotated gold labels. ## Dataset Structure The dataset is divided into three parts: - Training Set: Contains movie scripts and summaries with silver scene saliency labels. - Validation Set: Contains movie scripts and summaries with human-annotated gold scene saliency labels. - Test Set: Contains movie scripts and summaries with human-annotated gold scene saliency labels.

MS^2

MS^2 (Multi-Document Summarization of Medical Studies) is a dataset of over 470k documents and 20k summaries derived from the scientific literature. This dataset facilitates the development of systems that can assess and aggregate contradictory evidence across multiple studies, and is one of the first large-scale, publicly available multi-document summarization dataset in the biomedical domain.

SPACE (Opinion Summarization)

SPACE is a large-scale opinion summarization benchmark for the evaluation of unsupervised summarizers. SPACE is built on TripAdvisor hotel reviews and includes a training set of approximately 1.1 million reviews for over 11 thousand hotels. For evaluation, we created a collection of human-written, abstractive opinion summaries for 50 hotels, including high-level general summaries and aspect summaries for six popular aspects: building, cleanliness, food, location, rooms, and service. Every summary is based on 100 input reviews, an order of magnitude increase compared to existing corpora. In total, SPACE contains 1,050 gold standard summaries. You can view the full instructions for out multi-stage annotation procedure here.

Wikipedia Generation

Wikipedia Generation is a dataset for article generation from Wikipedia from references at the end of Wikipedia page and the top 10 search results for the Wikipedia topic.

CHQ-Summ

Contains 1507 domain-expert annotated consumer health questions and corresponding summaries. The dataset is derived from the community question answering forum and therefore provides a valuable resource for understanding consumer health-related posts on social media.

CNewSum

CNewSum is a large-scale Chinese news summarization dataset which consists of 304,307 documents and human-written summaries for the news feed. It has long documents with high-abstractive summaries, which can encourage document-level understanding and generation for current summarization models. An additional distinguishing feature of CNewSum is that its test set contains adequacy and deducibility annotations for the summaries.

DMQA

The DeepMind Q&A Dataset consists of two datasets for Question Answering, CNN and DailyMail. Each dataset contains many documents (90k and 197k each), and each document companies on average 4 questions approximately. Each question is a sentence with one missing word/phrase which can be found from the accompanying document/context.

Healthline

Healthline is a nutrition related dataset for multi-document summarization, using scientific studies.

PerKey

A corpus of 553k news articles from six Persian news websites and agencies with relatively high quality author extracted keyphrases, which is then filtered and cleaned to achieve higher quality keyphrases. Source: PerKey: A Persian News Corpus for Keyphrase Extraction and Generation

Shmoop Corpus

Shmoop Corpus is a dataset of 231 stories that are paired with detailed multi-paragraph summaries for each individual chapter (7,234 chapters), where the summary is chronologically aligned with respect to the story chapter. From the corpus, a set of common NLP tasks are constructed, including Cloze-form question answering and a simplified form of abstractive summarization, as benchmarks for reading comprehension on stories. Source: Shmoop Corpus

TextBox 2.0

TextBox 2.0 is a comprehensive and unified library for text generation, focusing on the use of pre-trained language models (PLMs). The library covers 13 common text generation tasks and their corresponding 83 datasets and further incorporates 45 PLMs covering general, translation, Chinese, dialogue, controllable, distilled, prompting, and lightweight PLMs. Source: TextBox 2.0: A Text Generation Library with Pre-trained Language Models Image Source: https://arxiv.org/pdf/2212.13005v1.pdf

CrossSum-IN

Given an English article, generate a short summary in the target language.

DaNewsroom

The first large-scale non-English language dataset specifically curated for automatic summarisation. The document-summary pairs are news articles and manually written summaries in the Danish language.

DUC 2007

There is currently much interest and activity aimed at building powerful multi-purpose information systems. The agencies involved include DARPA, ARDA and NIST. Their programmes, for example DARPA's TIDES (Translingual Information Detection Extraction and Summarization) programme, ARDA's Advanced Question & Answering Program and NIST's TREC (Text Retrieval Conferences) programme cover a range of subprogrammes. These focus on different tasks requiring their own evaluation designs. Within TIDES and among other researchers interested in document understanding, a group grew up which has been focusing on summarization and the evaluation of summarization systems. Part of the initial evaluation for TIDES called for a workshop to be held in the fall of 2000 to explore different ways of summarizing a common set of documents. Additionally a road mapping effort was started in March of 2000 to lay plans for a long-term evaluation effort in summarization. Out of the initial workshop and the roadmapping effort has grown a continuing evaluation in the area of text summarization called the Document Understanding Conferences (DUC). Sponsored by the Advanced Research and Development Activity (ARDA), the conference series is run by the National Institute of Standards and Technology (NIST) to further progress in summarization and enable researchers to participate in large-scale experiments.

Elsevier OA CC-BY

An open corpus of Scientific Research papers which has a representative sample from across scientific disciplines. This corpus not only includes the full text of the article, but also the metadata of the documents, along with the bibliographic information for each reference. Source: Elsevier OA CC-By Corpus

FIB

Factual Inconsistency Benchmark (FIB) is a benchmark that focuses on the task of summarization. Specifically, the benchmark involves comparing the scores an LLM assigns to a factually consistent versus a factual inconsistent summary for an input news article. For factually consistent summaries, human-written reference summaries are used to manually verify as factually consistent. Source: Evaluating the Factual Consistency of Large Language Models Through Summarization Image Source: https://arxiv.org/pdf/2211.08412v1.pdf

HumSet

Timely and effective response to humanitarian crises requires quick and accurate analysis of large amounts of text data, a process that can highly benefit from expert-assisted NLP systems trained on validated and annotated data in the humanitarian response domain. To enable creation of such NLP systems, we introduce and release HumSet, a novel and rich multilingual dataset of humanitarian response documents annotated by experts in the humanitarian response community. The dataset provides documents in three languages (English, French, Spanish) and covers a variety of humanitarian crises from 2018 to 2021 across the globe. For each document, HumSet provides selected snippets (entries) as well as assigned classes to each entry annotated using common humanitarian information analysis frameworks. HumSet also provides novel and challenging entry extraction and multi-label entry classification tasks. In this paper, we take a first step towards approaching these tasks and conduct a set of experiments on Pre-trained Language Models (PLM) to establish strong baselines for future research in this domain. The dataset is available at https://blog.thedeep.io/humset/.

MentSum

Mental health remains a significant challenge of public health worldwide. With increasing popularity of online platforms, many use the platforms to share their mental health conditions, express their feelings, and seek help from the community and counselors. While posts are of varying length, it is beneficial to provide a short, but informative summary for fast processing by the counselors. To facilitate research in summarization of mental health online posts, we introduce Mental Health Summarization dataset, MentSum, containing over 24k carefully selected user posts from Reddit, along with their short user-written summary (called TLDR) in English from 43 mental health subreddits. MentSum is distributed through Data Usage Agreement (DUA). Please fill out the request form at https://ir.cs.georgetown.edu/resources/ to obtain this dataset.

OpenAsp

OpenAsp Dataset OpenAsp is an Open Aspect-based Multi-Document Summarization dataset derived from DUC and MultiNews summarization datasets. Antarctica ## Dataset Access To generate OpenAsp, you require access to the DUC dataset which OpenAsp is derived from. ### Steps: 1. Grant access to DUC dataset by following NIST instructions here. * you should receive two user-password pairs (for DUC01-02 and DUC06-07) * you should receive a file named fwdrequestingducdata.zip 2. Clone this repository by running the following command: git clone https://github.com/liatschiff/OpenAsp.git 3. Optionally create a conda or virtualenv environment: bash conda create -n openasp 'python>3.10,<3.11' conda activate openasp 4. Install python requirements, currently requires python3.8-3.10 (later python versions have issues with spacy) bash pip install -r requirements.txt 5. copy fwdrequestingducdata.zip into the OpenAsp repo directory 6. run the prepare script command: bash python prepare_openasp_dataset.py --nist-duc2001-user '<2001-user>' --nist-duc2001-password '<2001-pwd>' --nist-duc2006-user '<2006-user>' --nist-duc2006-password '<2006-pwd>' 7. load the dataset using huggingface datasets python from glob import glob import os import gzip import shutil from datasets import load_dataset openasp_files = os.path.join('openasp-v1', '*.jsonl.gz') data_files = { os.path.basename(fname).split('.')[0]: fname for fname in glob(openasp_files) } for ftype, fname in data_files.copy().items(): with gzip.open(fname, 'rb') as gz_file: with open(fname[:-3], 'wb') as output_file: shutil.copyfileobj(gz_file, output_file) data_files[ftype] = fname[:-3] # load OpenAsp as huggingface's dataset openasp = load_dataset('json', data_files=data_files) # print first sample from every split for split in ['train', 'valid', 'test']: sample = openasp[split][0] # print title, aspect_label, summary and documents for the sample title = sample['title'] aspect_label = sample['aspect_label'] summary = '\n'.join(sample['summary_text']) input_docs_text = ['\n'.join(d['text']) for d in sample['documents']] print('* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *') print(f'Sample from {split}\nSplit title={title}\nAspect label={aspect_label}') print(f'\naspect-based summary:\n {summary}') print('\ninput documents:\n') for i, doc_txt in enumerate(input_docs_text): print(f'---- doc #{i} ----') print(doc_txt[:256] + '...') print('* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *\n\n\n') ### Troubleshooting 1. Dataset failed loading with load_dataset() - you may want to delete huggingface datasets cache folder 2. 401 Client Error: Unauthorized - you're DUC credentials are incorrect, please verify them (case sensitive, no extra spaces etc) 3. Dataset created but prints a warning about content verification - you may be using different version of NLTK or spacy model which affects the sentence tokenization process. You must use exact versions as pinned on requirements.txt. 4. IndexError: list index out of range - similar to (3), try to reinstall the requirements with exact package versions. ### Under The Hood The prepare_openasp_dataset.py script downloads DUC and Multi-News source files, uses sacrerouge package to prepare the datasets and uses the openasp_v1_dataset_metadata.json file to extract the relevant aspect summaries and compile the final OpenAsp dataset. ## License This repository, including the openasp_v1_dataset_metadata.json and prepare_openasp_dataset.py, are released under APACHE license. OpenAsp dataset summary and source document for each sample, which are generated by running the script, are licensed under the respective generic summarization dataset - Multi-News license and DUC license.

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.