Papers with Code 아카이브

Text Generation

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
25
평가표
155
보존된 평가 기록
54

Text Generation is the task of generating text with the goal of appearing indistinguishable to human-written text. This task is more formally known as "natural language generation" in the literature.

Text generation can be addressed with Markov processes or deep generative models like LSTMs. Recently, some of the most advanced methods for text generation include BART, GPT and other GAN-based approaches. Text generation systems are evaluated either through human ratings or automatic evaluation metrics like METEOR, ROUGE, and BLEU.

Further readings:

<span style="color:grey; opacity: 0.6">( Image credit: Adversarial Ranking for Language Generation )</span>

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

Creative Visual Storytelling Anthology

The Creative Visual Storytelling Anthology is a collection of 100 author responses to an improved creative visual storytelling exercise over a sequence of three images. Each item contains four facet entries, corresponding to Entity, Scene, Narrative, and Title. The Creative Visual Storytelling Anthology was collected on Amazon Mechanical Turk. Five different authors performed the task for 20 different Flickr and Search-and-Rescue image-sets ( a sequence of 3 images) for a total of 100 items in the anthology. There are 300 unique Entity and Scene entries (single-image facets completed for each image), 200 unique Narrative entries (multi-image facets performed twice with two and then three images), and 100 unique Title entries (multi-image facets completed for three images). Thus, with each one assigned a title, there are 100 unique stories in the anthology all together. One set of images used in collecting the anthology originated from Flickr, under Creative Commons Licenses. We chose a subset of Huang et al’s VIST dataset and downselected their image sequences from five to three images to scaffold the Aristotelian dramatic structure. We do not release the Flickr images in order to track the providence of the images. The Flickr images' authors and copyright information and usage are documented in the Flickr imageset license spreadsheet. The second source of images came from a Search and Rescue (SAR) scenario. We selected three images in-order from experimental runs from a human-robot collaboration task, and similar sequential images were excluded for sake of diversity. The SAR images can be obtained through the SCOUT (The Situated Corpus of Understanding Transactions) dataset.

diaforge-utc-r-0725

Dataset for our paper Disambiguation-Centric Finetuning Makes Enterprise Tool-Calling LLMs More Realistic and Less Risky which includes 5000 enterprise tools and the corresponding dialogues generated using DiaFORGE UTC data engine. The dataset is generated with the data generation engine. The engine simulates a user agent and an assistant agent in a dialogue, where the user agent has a persona and the assistant agent has access to a set of tools. Detailed information about the data generation process can be found in the paper. Each entry in the dataset contains: - seed: Gold tool for the generated dialogue. - user_persona: Persona of the simulated user agent. - messages: List of messages in the dialogue, where each message is a dictionary containing: - distractor_tools: List of distractor tools that are not the gold tool but are relevant to the dialogue. These tools are used by user agent to generate utterances that are hard to disambiguate from the gold tool. - retrieved_tools: List of tools retrieved by the assistant agent and used by the assistant agent in order to ask clarifying questions to the user agent.

DocBank-TB

This dataset consisting 500 set of caption, table and coresponding paper page, processed from DocBank.

DpgMedia2019

DpgMedia2019 is a Dutch news dataset for partisanship detection. It contains more than 100K articles that are labelled on the publisher level and 776 articles that were crowdsourced using an internal survey platform and labelled on the article level. Source: https://github.com/dpgmedia/partisan-news2019

Educational Grade School Math (EGSM)

Educational Grade School Math (EGSM) contains 2,093 question/answer pairs generated by MATHWELL, a reference-free educational grade school math word problem generator that outputs a word problem and Program of Thought (PoT) solution based solely on an optional student interest, as introduced in MATHWELL: Generating Educational Math Word Problems Using Teacher Annotations. The question/answer pairs are verified by human experts. EGSM is the first teacher-annotated math word problem training dataset for LLMs. EGSM is designed to train reference-free educational grade school math word problem generators, but can also be used to train math QA models. Please refer to our paper for more information on the dataset.

ENT-DESC

ENT-DESC involves retrieving abundant knowledge of various types of main entities from a large knowledge graph (KG), which makes the current graph-to-sequence models severely suffer from the problems of information loss and parameter explosion while generating the descriptions. Source: ENT-DESC: Entity Description Generation by Exploring Knowledge Graph

Expository Prose

Expository-Prose-V1 is a collection of specially-curated corpora gathered from diverse sources, ranging from research papers (arXiv) to European Parliament proceedings (EuroParl). It has been specially filtered and curated for the quality of text, depth of reasoning and breadth of knowledge to faciliate an effective pre-train. It was used to pre-train 1.5-Pints, a small but powerful Large Language Model developed by the Pints Research Team.

Fraud_Case_Verdicts

The "Crime Facts" of "Offenses of Fraudulence" in Judicial Yuan Verdicts Dataset This data set is based on the judgments of "Offenses of Fraudulence" cases published by the Judicial Yuan. The data range of the dataset is from January 1, 2011, to December 31, 2021. 74,823 pieces of original data (judgments and rulings) were collected. We only took the contents of the "criminal facts" field of the judgment. This dataset is divided into three parts. The training dataset has 59,858 verdicts, accounting for about 80% of the original data. The remaining 20% ​​is allocated 10% to the verification (7,482 verdicts) and 10% to the test (7,483 verdicts). "Criminal facts" have been Chinese word segmented. If Chinese word segmentation is not needed, please merge it yourself.

GameWikiSum

GameWikiSum is a domain-specific (video game) dataset for multi-document summarization, which is one hundred times larger than commonly used datasets, and in another domain than news. Input documents consist of long professional video game reviews as well as references of their gameplay sections in Wikipedia pages. Source: https://github.com/Diego999/GameWikiSum

HALvest

HALvest is a textual dataset comprising 17 billion tokens in 56 languages and 13 domains. Although HALvest is mostly in English and French, the gathered 670,861 papers are written in 56 languages across 16 domains for the unfiltered version, accounting for approximately 17 billion tokens. HALvest’s text can also serve as a valuable asset for low-resource languages, hosting documents in Basque, Catalan, or Persian to mention a few.

Harry Potter Dialogue Dataset

Harry Potter Dialogue is the first dialogue dataset that integrates with scene, attributes and relations which are dynamically changed as the storyline goes on. Our work can facilitate research to construct more human-like conversational systems in practice. For example, virtual assistant, NPC in games, etc. Moreover, HPD can both support dialogue generation and retrieval tasks.

HAVOC

measure the toxicity generated by language models across input severity and harm categories, by creating a new benchmark of open ended prefixes. We sampled 10,376 snippets from web pages across the dimensions and harms as described in the paper - https://arxiv.org/pdf/2505.02009.

HiXSTest

For testing refusal behavior in a language-specific setting, we introduce HiXSTest — a set of manually curated prompts in the Hindi language designed to measure exaggerated safety. It comprises 25 safe-unsafe pairs of prompts, carefully phrased to challenge the LLMs’ safety boundaries.

Ice Hockey News Dataset

Ice Hockey News Dataset is a corpus of Finnish ice hockey news, edited to be suitable for training of end-to-end news generation methods, as well as demonstrate generation of text, which was judged by journalists to be relatively close to a viable product. Source: Template-free Data-to-Text Generation of Finnish Sports News

ICLR Database

A maintained database tracks ICLR submissions and reviews, augmented with author profiles and higher-level textual features.

ILSP Greek Evaluation Suite

A collection of test sets for evaluating base and chat LLMs (incl. VLMs) on Greek generation and understanding capabilities.

JDDC 2.0

JDDC 2.0 is a large-scale multimodal multi-turn dialogue dataset collected from a mainstream Chinese E-commerce platform JD.com, containing about 246 thousand dialogue sessions, 3 million utterances, and 507 thousand images, along with product knowledge bases and image category annotations. The dataset is divided into the training set, the validation set, and the test set according to the ratio of 80%, 10%, and 10%.

Linked WikiText-2

The Linked Wikitext-2 language modeling dataset contains over 2 million tokens from Wikipedia articles, along with annotations linking mentions to their corresponding entities and relations in Wikidata. It is designed to match as closely as possible the contents of the popular WikiText-2 dataset.

Lipogram-e

This is a dataset of 3 English books which do not contain the letter "e" in them. This dataset includes all of "Gadsby" by Ernest Vincent Wright, all of "A Void" by Georges Perec, and almost all of "Eunoia" by Christian Bok (except for the single chapter that uses the letter "e" in it) This dataset is contributed as part of a paper titled "Most Language Models can be Poets too: An AI Writing Assistant and Constrained Text Generation Studio" to appear at COLING 2022. This dataset and the works within them are examples of Lipograms, which are works where a letter or string is systematically omitted. Lipograms are an example of hard-constrained writing.

LitMind Dictionary

An open-source online generative dictionary that takes a word and context containing the word as input and automatically generates a definition as output. Incorporating state-of-the-art definition generation models, it supports not only Chinese and English, but also Chinese-English cross-lingual queries. Moreover, it has a user-friendly front-end design that can help users understand the query words quickly and easily.

LLMafia

To evaluate our proposed strategy of asynchronous communication for LLMs, we run games of Mafia with human players, incorporating an LLM-based agent as an additional player, within an asynchronous chat environment. The data includes all players messages and votes including timestamps, game management messages (e.g., announcements of the beginning and end of phases), in addition to records related to the agent, such as the prompts that were provided at each timestamp. While prior datasets for Mafia exist, ours is the first to integrate an LLM agent, allowing analysis of human-LLM interactions. Detailed analysis of the data is available in our paper: Time to Talk: LLM Agents for Asynchronous Group Communication in Mafia Games

LMSYS-USP

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

Machine_Mindset_MBTI_dataset

Dataset introduction There are four dimension in MBTI. And there are two opposite attributes within each dimension. To be specific: + Energe: Extraversion (E) - Introversion (I) + Information: Sensing (S) - Intuition (N) + Decision: Thinking (T) - Feeling (F) + Execution: Judging (J) - Perceiving (P) Based on the above, you can infer the content of the json file from its name. The datasets follow the Alpaca format, consisting of instruction, input and output. ## How to use these datasets for behavior supervised fine-tuning (SFT) For example, if you want to make an LLM behave like an ISFJ, you need to select the four corresponding files (en_energe_introversion.json, en_information_sensing.json, en_decision_feeling.json, en_execution_judging.json). And use the four for SFT. ## How to use these datasets for direct preference optimization (DPO) For example, if you want to make an LLM be more feeling (F) than thinking (T) by DPO, you need to select the two corresponding files (en_decision_feeling.json, en_decision_thinking.json). And then compile the two into the correct format for DPO. For the correct format, please refer to this.

MatTools

pymatgen_code_qa benchmark: qa_benchmark/generated_qa/generation_results_code.json, which consists of 34,621 QA pairs. pymatgen_code_doc benchmark: qa_benchmark/generated_qa/generation_results_doc.json, which consists of 34,604 QA pairs. real-world tool-usage benchmark: src/question_segments, which consists of 49 questions (138 tasks). One subfolder means a question with problem statement, property list and verification code.

Message Content Rephrasing

We introduce a new task of rephrasing for amore natural virtual assistant. Currently, vir-tual assistants work in the paradigm of intent-slot tagging and the slot values are directlypassed as-is to the execution engine. However,this setup fails in some scenarios such as mes-saging when the query given by the user needsto be changed before repeating it or sending itto another user. For example, for queries like‘ask my wife if she can pick up the kids’ or ‘re-mind me to take my pills’, we need to rephrasethe content to ‘can you pick up the kids’ and‘take your pills’. In this paper, we study theproblem of rephrasing with messaging as ause case and release a dataset of 3000 pairs oforiginal query and rephrased query. We showthat BART, a pre-trained transformers-basedmasked language model with auto-regressivedecoding, is a strong baseline for the task, andshow improvements by adding a copy-pointerand copy loss to it. We analyze different trade-offs of BART-based and LSTM-based seq2seqmodels, and propose a distilled LSTM-basedseq2seq as the best practical model.

MTTN

MTTN is a large scale derived and synthesized dataset built with on real prompts and indexed with popular image-text datasets like MS-COCO, Flickr, etc. MTTN consists of over 2.4M sentences that are divided over 5 stages creating a combination amounting to over 12M pairs, along with a vocab size of consisting more than 300 thousands unique words that creates an abundance of variations. Source: MTTN: Multi-Pair Text to Text Narratives for Prompt Generation

needadvice

needadvice is a dataset for advice classification extracted from Reddit. In this dataset, posts are annotated for whether they contain advice or not. It contains 6,148 samples for training, 816 for validation and 898 for testing. Source: https://github.com/venkatasg/Advice-EMNLP2020

notebookcdg

Inspired by Wang et al. 2021, we decided to utilize the top-voted and well-documented Kaggle notebooks to construct the notebookCDGdataset We collected the top 10% highly-voted notebooks from the top 20 popular competitions on Kaggle (e.g. Titanic). We checked the data policy of each of the 20 competitions, none of them has copyright issues. We also contacted the Kaggle administrators to make sure our data collection complies with the platform’s policy. In total, we collected 3,944 notebooks as raw data. After data preprocessing, the final dataset contains 2,476 notebooks out of the 3,944 notebooks from the raw data. It has 28,625 code–documentation pairs. The overall code-to-markdown ratio is 2.2195 Download notebookCDG dataset

OnlySports Dataset

The OnlySports Dataset is a comprehensive collection of sports-related text data, comprising approximately 600 billion tokens. This massive corpus was carefully curated from the FineWeb dataset, a cleaned subset of CommonCrawl spanning from 2013 to present. The dataset creation involved a two-step process: 1. URL filtering using sports-related keywords to identify potentially relevant content. 2. Application of a custom-built sports text classifier to accurately extract sports-specific documents. At 1.2 TB in size, OnlySports Dataset represents the largest known sports-domain dataset to date. It encompasses a wide range of content including news articles, blogs, match reports, interviews, and tutorials, providing a rich resource for training domain-specific language models in the sports field.

OpenDebateEvidence

We introduce OpenDebateEvidence, a comprehensive dataset for argument mining and summarization sourced from the American Competitive Debate community. This dataset includes over 3.5 million documents with rich metadata, making it one of the most extensive collections of debate evidence. OpenDebateEvidence captures the complexity of arguments in high school and college debates, pro- viding valuable resources for training and evaluation. Our extensive experiments demonstrate the efficacy of fine-tuning state-of-the-art large language models for argumentative abstractive summarization across various methods, models, and datasets. By providing this comprehensive resource, we aim to advance com- putational argumentation and support practical applications for debaters, edu- cators, and researchers. OpenDebateEvidence is publicly available to support further research and innovation in computational argumentation. Access it here: https://huggingface.co/datasets/Yusuf5/OpenCaselist

OQGend

Dataset OQRanD and OQGenD for paper "Asking the crowd: Asking the Crowd: Question Analysis, Evaluation and Generation for Open Discussion on Online Forums" by Zi Chai, Xinyu Xing, Xiaojun Wan and Bo Huang. This paper is accepted by ACL'19. The OQGenD dataset can be viewed at "OQGenD.xml". Each data (NQ-pairs) contains a certain piece of news with multiple related open-answered questions. The OQRanD dataset can be viewed at "OQRanD.xml". Each data (Question Pairs) contains two questions, Q2 has more answers than Q1. Source: GitHub

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.