Papers with Code · CC BY-SA 4.0

데이터셋 탐색

데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.

관련 논문 8편

Web Traffic Time Series Forecasting

Wiki
Wiki 설명 보기

Context There's a story behind every dataset and here's your opportunity to share yours. ### Content What's inside is more than just rows and columns. Make it easy for others to get started by describing how you acquired the data and what time period it represents, too. ### Acknowledgements We wouldn't be here without the help of others. If you owe any attributions or thanks, include them here along with any citations of past research. ### Inspiration Your data will be in front of the world's largest data science community. What questions do you want to see answered?

관련 논문 1,081편

WikiText-2

WikiText-2 설명 보기

The WikiText language modeling dataset is a collection of over 100 million tokens extracted from the set of verified Good and Featured articles on Wikipedia. The dataset is available under the Creative Commons Attribution-ShareAlike License. Compared to the preprocessed version of Penn Treebank (PTB), WikiText-2 is over 2 times larger and WikiText-103 is over 110 times larger. The WikiText dataset also features a far larger vocabulary and retains the original case, punctuation and numbers - all of which are removed in PTB. As it is composed of full articles, the dataset is well suited for models that can take advantage of long term dependencies. Source: The WikiText Long Term Dependency Language Modeling Dataset Image Source: https://blog.einstein.ai/the-wikitext-long-term-dependency-language-modeling-dataset/

관련 논문 560편

WikiText-103

WikiText-103 설명 보기

The WikiText language modeling dataset is a collection of over 100 million tokens extracted from the set of verified Good and Featured articles on Wikipedia. The dataset is available under the Creative Commons Attribution-ShareAlike License. Compared to the preprocessed version of Penn Treebank (PTB), WikiText-2 is over 2 times larger and WikiText-103 is over 110 times larger. The WikiText dataset also features a far larger vocabulary and retains the original case, punctuation and numbers - all of which are removed in PTB. As it is composed of full articles, the dataset is well suited for models that can take advantage of long term dependencies. Source: The WikiText Long Term Dependency Language Modeling Dataset Image Source: https://blog.einstein.ai/the-wikitext-long-term-dependency-language-modeling-dataset/

관련 논문 267편

WikiSQL

WikiSQL 설명 보기

WikiSQL consists of a corpus of 87,726 hand-annotated SQL query and natural language question pairs. These SQL queries are further split into training (61,297 examples), development (9,145 examples) and test sets (17,284 examples). It can be used for natural language inference tasks related to relational databases. Source: SQL-to-Text Generation with Graph-to-Sequence Model Image Source: https://blog.einstein.ai/how-to-talk-to-your-database/

관련 논문 208편

Wikipedia Squirrel

Wiki Squirrel
Wiki Squirrel 설명 보기

The data was collected from the English Wikipedia (December 2018). These datasets represent page-page networks on specific topics (chameleons, crocodiles and squirrels). Nodes represent articles and edges are mutual links between them. The edges csv files contain the edges - nodes are indexed from 0. The features json files contain the features of articles - each key is a page id, and node features are given as lists. The presence of a feature in the feature list means that an informative noun appeared in the text of the Wikipedia article. The target csv contains the node identifiers and the average monthly traffic between October 2017 and November 2018 for each page. For each page-page network we listed the number of nodes an edges with some other descriptive statistics.

관련 논문 196편

Wikipedia open-domain Question Answering

WikiQA
WikiQA 설명 보기

The WikiQA corpus is a publicly available set of question and sentence pairs, collected and annotated for research on open-domain question answering. In order to reflect the true information need of general users, Bing query logs were used as the question source. Each question is linked to a Wikipedia page that potentially has the answer. Because the summary section of a Wikipedia page provides the basic and usually most important information about the topic, sentences in this section were used as the candidate answers. The corpus includes 3,047 questions and 29,258 sentences, where 1,473 sentences were labeled as answer sentences to their corresponding questions. Source: http://aka.ms/WikiQA Image Source: Yang et al

관련 논문 145편

Wizard of Wikipedia

Wizard of Wikipedia 설명 보기

Wizard of Wikipedia is a large dataset with conversations directly grounded with knowledge retrieved from Wikipedia. It is used to train and evaluate dialogue systems for knowledgeable open dialogue with clear grounding

관련 논문 111편

Wiki-CS

Wiki-CS 설명 보기

Wiki-CS is a Wikipedia-based dataset for benchmarking Graph Neural Networks. The dataset is constructed from Wikipedia categories, specifically 10 classes corresponding to branches of computer science, with very high connectivity. The node features are derived from the text of the corresponding articles. They were calculated as the average of pretrained GloVe word embeddings (Pennington et al., 2014), resulting in 300-dimensional node features. The dataset has 11,701 nodes and 216,123 edges. Source: Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks Image Source: https://arxiv.org/pdf/2007.02901v1.pdf

관련 논문 102편

2WikiMultiHopQA

2WikiMultiHopQA 설명 보기

Uses structured and unstructured data. The dataset introduces the evidence information containing a reasoning path for multi-hop questions. Source: Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps

관련 논문 91편

WikiMatrix

WikiMatrix 설명 보기

WikiMatrix is a dataset of parallel sentences in the textual content of Wikipedia for all possible language pairs. The mined data consists of: - 85 different languages, 1620 language pairs - 134M parallel sentences, out of which 34M are aligned with English Source: WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia

관련 논문 84편

Wikipedia Biography Dataset

WikiBio
WikiBio 설명 보기

This dataset gathers 728,321 biographies from English Wikipedia. It aims at evaluating text generation algorithms. For each article, we provide the first paragraph and the infobox (both tokenized).

관련 논문 79편

Wikipedia-based Image Text

WIT
WIT 설명 보기

Wikipedia-based Image Text (WIT) Dataset is a large multimodal multilingual dataset. WIT is composed of a curated set of 37.6 million entity rich image-text examples with 11.5 million unique images across 108 Wikipedia languages. Its size enables WIT to be used as a pretraining dataset for multimodal machine learning models. Key Advantages A few unique advantages of WIT: - The largest multimodal dataset (time of this writing) by the number of image-text examples. - A massively multilingual (first of its kind) with coverage for over 100+ languages. - A collection of diverse set of concepts and real world entities. - Brings forth challenging real-world test sets.

관련 논문 79편

WikiTableQuestions

WikiTableQuestions 설명 보기

WikiTableQuestions is a question answering dataset over semi-structured tables. It is comprised of question-answer pairs on HTML tables, and was constructed by selecting data tables from Wikipedia that contained at least 8 rows and 5 columns. Amazon Mechanical Turk workers were then tasked with writing trivia questions about each table. WikiTableQuestions contains 22,033 questions. The questions were not designed by predefined templates but were hand crafted by users, demonstrating high linguistic variance. Compared to previous datasets on knowledge bases it covers nearly 4,000 unique column headers, containing far more relations than closed domain datasets and datasets for querying knowledge bases. Its questions cover a wide range of domains, requiring operations such as table lookup, aggregation, superlatives (argmax, argmin), arithmetic operations, joins and unions. Source: Explaining Queries over Web Tables to Non-Experts Image Source: https://ppasupat.github.io/WikiTableQuestions/

관련 논문 73편

IMDB-WIKI

IMDB-WIKI 설명 보기

To the best of our knowledge this is the largest publicly available dataset of face images with gender and age labels for training. We provide pretrained models for both age and gender prediction.

관련 논문 72편

PAN-X

WikiANN
WikiANN 설명 보기

WikiANN, also known as PAN-X, is a multilingual named entity recognition dataset. It consists of Wikipedia articles that have been annotated with LOC (location), PER (person), and ORG (organization) tags in the IOB2 format¹². This dataset serves as a valuable resource for training and evaluating named entity recognition models across various languages. For instance, it includes information about notable individuals, places, and organizations mentioned in Wikipedia articles. Researchers and practitioners can use WikiANN to develop and improve natural language processing systems that identify and classify named entities in text. (1) wikiann · Datasets at Hugging Face. https://huggingface.co/datasets/wikiann. (2) wikiann | TensorFlow Datasets. https://tensorflow.google.cn/datasets/catalog/wikiann. (3) wikiann · Datasets at Hugging Face. https://huggingface.co/datasets/wikiann/viewer/en. (4) WikiAnn Dataset | Papers With Code. https://paperswithcode.com/dataset/wikiann-1.

관련 논문 66편

WikiHop

WikiHop 설명 보기

WikiHop is a multi-hop question-answering dataset. The query of WikiHop is constructed with entities and relations from WikiData, while supporting documents are from WikiReading. A bipartite graph connecting entities and documents is first built and the answer for each query is located by traversal on this graph. Candidates that are type-consistent with the answer and share the same relation in query with the answer are included, resulting in a set of candidates. Thus, WikiHop is a multi-choice style reading comprehension data set. There are totally about 43K samples in training set, 5K samples in development set and 2.5K samples in test set. The test set is not provided. The task is to predict the correct answer given a query and multiple supporting documents. The dataset includes a masked variant, where all candidates and their mentions in the supporting documents are replaced by random but consistent placeholder tokens. Source: Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs Image Source: http://qangaroo.cs.ucl.ac.uk/

관련 논문 54편

WikiSum

WikiSum 설명 보기

WikiSum is a dataset based on English Wikipedia and suitable for a task of multi-document abstractive summarization. In each instance, the input is comprised of a Wikipedia topic (title of article) and a collection of non-Wikipedia reference documents, and the target is the Wikipedia article text. The dataset is restricted to the articles with at least one crawlable citation. The official split divides the articles roughly into 80/10/10 for train/development/test subsets, resulting in 1865750, 233252, and 232998 examples respectively. Source: Generating Wikipedia by Summarizing Long Sequences Image Source: https://arxiv.org/pdf/1801.10198.pdf

관련 논문 54편

Wikidata5M

Wikidata5M 설명 보기

Wikidata5m is a million-scale knowledge graph dataset with aligned corpus. This dataset integrates the Wikidata knowledge graph and Wikipedia pages. Each entity in Wikidata5m is described by a corresponding Wikipedia page, which enables the evaluation of link prediction over unseen entities. The dataset is distributed as a knowledge graph, a corpus, and aliases. We provide both transductive and inductive data splits used in the original paper.

관련 논문 31편

Wikipedia Current Events Portal

WCEP
WCEP 설명 보기

The WCEP dataset for multi-document summarization (MDS) consists of short, human-written summaries about news events, obtained from the Wikipedia Current Events Portal (WCEP), each paired with a cluster of news articles associated with an event. These articles consist of sources cited by editors on WCEP, and are extended with articles automatically obtained from the Common Crawl News dataset. Source: WCEP

관련 논문 26편

WikiReading

WikiReading 설명 보기

WikiReading is a large-scale natural language understanding task and publicly-available dataset with 18 million instances. The task is to predict textual values from the structured knowledge base Wikidata by reading the text of the corresponding Wikipedia articles. The task contains a rich variety of challenging classification and extraction sub-tasks, making it well-suited for end-to-end models such as deep neural networks (DNNs). Source: WIKIREADING: A Novel Large-scale Language Understanding Task over Wikipedia Image Source: Hewlett et al

관련 논문 24편

Wiki-ZSL

Wiki-ZSL 설명 보기

The Wiki-ZSL (Wiki Zero-Shot Learning) dataset contains 113 relations and 94,383 instances from Wikipedia. The dataset is divided into three subsets: training set (98 relations), validation set (5 relations) and test set (10 relations).

관련 논문 21편

WikiSplit

WikiSplit 설명 보기

Contains one million naturally occurring sentence rewrites, providing sixty times more distinct split examples and a ninety times larger vocabulary than the WebSplit corpus introduced by Narayan et al. (2017) as a benchmark for this task. Source: Learning To Split and Rephrase From Wikipedia Edit History

관련 논문 17편

WikiSection

WikiSection 설명 보기

A publicly available dataset with 242k labeled sections in English and German from two distinct domains: diseases and cities. Source: SECTOR: A Neural Model for Coherent Topic Segmentation and Classification

관련 논문 16편

Wiki-One

Wiki-One 설명 보기

This dataset is a Wikipedia dump, split by relations to perform Few-Shot Knowledge Graph Completion. \begin{table}[] \begin{tabular}{@{}lllccl@{}} \textbf{Dataset} & \textbf{# Ent} & \textbf{# Rel} & \textbf{# Triplets} & \textbf{Train/Dev/Test} \ Wiki-One & 4,838,244 & 822 & 5,829,240 & 133/16/34 \ \end{tabular} \caption{Datasets used in the experiments. } \end{table}

관련 논문 10편

WikiAtomicEdits

WikiAtomicEdits 설명 보기

WikiAtomicEdits is a corpus of 43 million atomic edits across 8 languages. These edits are mined from Wikipedia edit history and consist of instances in which a human editor has inserted a single contiguous phrase into, or deleted a single contiguous phrase from, an existing sentence. Source: WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discourse Image Source: https://arxiv.org/pdf/1808.09422v1.pdf

관련 논문 10편

WikiConv

WikiConv 설명 보기

A corpus that encompasses the complete history of conversations between contributors to Wikipedia, one of the largest online collaborative communities. By recording the intermediate states of conversations---including not only comments and replies, but also their modifications, deletions and restorations---this data offers an unprecedented view of online conversation. Source: WikiConv: A Corpus of the Complete Conversational History of a Large Online Collaborative Community

관련 논문 8편

WikiTableT

WikiTableT 설명 보기

WikiTableT contains Wikipedia article sections and their corresponding tabular data and various metadata. WikiTableT contains millions of instances while covering a broad range of topics and a variety of kinds of generation tasks. Source: WikiTableT

관련 논문 7편

WikiCatSum

WikiCatSum 설명 보기

WikiCatSum is a domain specific Multi-Document Summarisation (MDS) dataset. It assumes the summarisation task of generating Wikipedia lead sections for Wikipedia entities of a certain domain (e.g. Companies) from the set of documents cited in Wikipedia articles or returned by Google (using article titles as queries). The dataset includes three domains: Companies, Films, and Animals. Source: https://datashare.ed.ac.uk/handle/10283/3368

관련 논문 7편

WikiTables-TURL

WikiTables-TURL 설명 보기

The WikiTables-TURL dataset was constructed by the authors of TURL and is based on the WikiTable corpus, which is a large collection of Wikipedia tables. The dataset consists of 580,171 tables divided into fixed training, validation and testing splits. Additionally, the dataset contains metadata about each table, such as the table name, table caption and column headers. 406,706 of these tables are annotated for the Column Type Annotation (CTA) task, 55,970 tables for the Columns Property Annotation (CPA) task and 200,744 tables for the Cell Entity Annotation (CEA) task. As classes for the CTA and CPA, Freebase's types and relations were used, whereas for the CEA task entities from Freebase were used. The table below lists the total annotated columns (or cells in the case of CEA) for each split and for each task as well as the number of classes used for annotation. | | Training | Validation | Testing | Classes | |-----|--------|----------|-------|-------| | CTA | 628,254 |13,391| 13,025 | 255 | | CPA | 62,954| 2,175 | 2,072 | 121 | | CEA | 1,264,217 | 76,720 | 225,777 | 1,787,737 | The authors have made the dataset and its variants publicly available for download.

관련 논문 6편

French Wikipedia

French Wikipedia 설명 보기

French Wikipedia is a dataset used for pretraining the CamemBERT French language model. It uses the official 2019 French Wikipedia dumps

관련 논문 6편

WikiNews Arabic Diacritization Benchmark Dataset

WikiNews Dataset
WikiNews Dataset 설명 보기

The WikiNews Arabic Diacritization dataset is a test set composed of 70 WikiNews articles (majority are from 2013 and 2014) that cover a variety of themes, namely: politics, economics, health, science and technology, sports, arts, and culture. The articles are evenly distributed among the different themes (10 per theme). The articles contain 18,300 words with around 400 different sentences (Each line is considered as a sentence).

관련 논문 5편

WikiHowQA

WikiHowQA 설명 보기

WikiHowQA is a Community-based Question Answering dataset, which can be used for both answer selection and abstractive summarization tasks. It contains 76,687 questions in the train set, 8,000 in the development set and 22,354 in the test set. Source: WikiHowQA

관련 논문 5편

Wikipedia Person and Animal Dataset

Wikipedia Person and Animal Dataset 설명 보기

This dataset gathers 428,748 person and 12,236 animal infobox with descriptions based on Wikipedia dump (2018/04/01) and Wikidata (2018/04/12).

관련 논문 4편

InferWiki

InferWiki 설명 보기

InferWiki is a Knowledge Graph Completion (KGC) dataset that improves upon existing benchmarks in inferential ability, assumptions, and patterns. First, each testing sample is predictable with supportive data in the training set. Second, InferWiki initiates the evaluation following the open-world assumption and improves the inferential difficulty of the closed-world assumption, by providing manually annotated negative and unknown triples. Third, the dataset includes various inference patterns (e.g., reasoning path length and types) for comprehensive evaluation.

관련 논문 4편

TemporalWiki

TemporalWiki 설명 보기

TemporalWiki is a lifelong benchmark for ever-evolving LMs that utilizes the difference between consecutive snapshots of English Wikipedia and English Wikidata for training and evaluation, respectively. The benchmark hence allows researchers to periodically track an LM's ability to retain previous knowledge and acquire updated/new knowledge at each point in time.

관련 논문 4편

WIKIPerson

WIKIPerson 설명 보기

WIKIPerson is a high-quality human-annotated visual person linking dataset based on Wikipedia. The dataset contains a total of 48k different news images, covering 13k out of 120K Person named entities, each of which corresponds to a celebrity in Wikipedia. Unlike previously commonly-used datasets in EL, the mention in WIKIPerson is only an image containing the person entity with its bounding box. The corresponding label identifies a unique entity in Wikipedia. For each entity in the Wikipedia, we provide textual descriptions as well as images to satisfy the need of three sub-tasks. Source: Visual Named Entity Linking: A New Dataset and A Baseline Image Source: https://arxiv.org/pdf/2211.04872v1.pdf