Papers with Code 아카이브

Language Modelling

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
6
평가표
55
보존된 평가 기록
466

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

LatamXIX

A novel dataset of 19th-century Latin American press texts, which addresses the lack of specialized corpora for historical and linguistic analysis in this region.

ProteinGym

ProteinGym is a collection of benchmarks aiming at comparing the ability of models to predict the effects of protein mutations. The benchmarks in ProteinGym are divided according to mutation type (substitutions vs. indels), ground truth source (DMS assay vs. clinical annotation), and training regime (zero-shot vs. supervised). Pascal Notin, Aaron Kollasch, Daniel Ritter, Lood Van Niekerk, Steffanie Paul, Han Spinner, Nathan Rollins, Ada Shaw, Rose Orenbuch, Ruben Weitzman, Jonathan Frazer, Mafalda Dias, Dinko Franceschi, Yarin Gal, and Debora Marks https://github.com/OATML-Markslab/ProteinGym

RTC

RTC is a benchmark corpus of social media comments sampled over three years. The corpus consists of 36.36m unlabelled comments for adaptation and evaluation on an upstream masked language modelling task as well as 0.9m labelled comments for finetuning and evaluation on a downstream document classification task. The Reddit Time Corpus (RTC) covers three years between March 2017 and February 2020 and is split into 36 evenly-sized monthly subsets based on comment timestamps. RTC is sampled from the Pushshift Reddit dataset.

S-TEST

S-TEST is a benchmark for measuring the specificity of the language of pre-trained language models.

SART

SART is a collection of three datasets for Similarity, Analogies and Relatedness for the Tatar language. The three subsets are: * Similarity dataset - 202 pairs of words along with averaged human scores of similarity degree between the words (in 0-to-10 scale). For example, "йорт, бина, 7.69". * Relatedness dataset - 252 pairs of words along with averaged human scores of relatedness degree between the words. For example, "урам, балалар, 5.38". * Analogies dataset - set of analytical questions of the form A:B::C:D, meaning A to B as C to D, and D is to be predicted. For example, "Әнкара Төркия Париж Франция". Contains 34 categories, and in total 30 144 questions. Source: https://github.com/tat-nlp/SART

Sentimental LIAR

The Sentimental LIAR dataset is a modified and further extended version of the LIAR extension introduced by Kirilin et al. In this dataset, the multi-class labeling of LIAR is converted to a binary annotation by changing half-true, false, barely-true and pants-fire labels to False, and the remaining labels to True. Furthermore, the speaker names are converted to numerical IDs in order to avoid bias with regards to the textual representation of names. The binary-label dataset is then extended by adding sentiments derived using the Google NLP API. Sentiment analysis determines the overall attitude of the text (i.e., whether it is positive or negative), and is quantified by a numerical score. If the sentiment score is positive, then the sample is tagged as Positive for the sentiment attribute, otherwise Negative is assigned. A further extension is introduced by adding emotion scores extracted using the IBM NLP API for each claim, which determine the detected level of 6 emotional states namely anger, sadness, disgust, fear and joy. The score for each emotion is between the range of 0 and 1. Source: https://github.com/UNHSAILLab/SentimentalLIAR

SMC Text Corpus

Contents (As on March 4, 2019) -------- The text corpus contains running text from various free licensed sources. - The whole content of Malayalam Wikipedia extracted on January 1, 2019 - News/Article from various sources, source mentioned in respective files: - 251 Mb - 8,60,159 lines - 98,15,533 words - 10,11,11,885 characters The word corpus contains - Classified lexicon prepared for Malaylam Morphology Analyser project - Unique words extracted from Malayalam Wikipedia, Wictionary etc. - 14,27,392 words

Stack Exchange

The Stack Exchange dataset is a collection of data from various Stack Exchange sites, including Stack Overflow, Mathematics, Super User, and many others. It includes questions, answers, comments, tags, and other related data from these sites. The dataset is updated regularly and can be accessed through the Stack Exchange Data Explorer. It's also hosted by the Internet Archive and is updated every three months. Additionally, it's available on platforms like Google BigQuery and Kaggle.

Alexa Point of View

The Alexa Point of View dataset is point of view conversion dataset, a parallel corpus of messages spoken to a virtual assistant and the converted messages for delivery. The dataset contains parallel corpus of input (input column) message and POV converted messages (output column). An example of a pair is tell @CN@ that i'll be late [\t] hi @CN@, @SCN@ would like you to know that they'll be late. The input and pov-converted output pair is tab separated. @CN@ tag is a placeholder for the contact name (receiver) and @SCN@ tag is a placeholder for source contact name (sender). The total dataset has 46563 pairs. This data is then test/train/dev split into 6985 pairs/32594 pairs/6985 pairs. Source: https://github.com/alexa/alexa-point-of-view-dataset

blbooks

This dataset consists of books digitised by the British Library in partnership with Microsoft. The dataset includes ~25 million pages of out of copyright texts. The majority of the texts were published in the 18th and 19th Century, but the collection also consists of a smaller number of books from earlier periods. Items within this collection cover a wide range of subject areas, including geography, philosophy, history, poetry and literature and are published in various languages. While the books are predominately from the 18th and 19th Centuries, there are fewer books from earlier periods.

Books3

The Books3 dataset emerged as part of a broader effort to train AI models for natural language understanding and generation. It comprises an extensive collection of digitized books, spanning from classics to contemporary works. These books were gathered from various sources, including libraries and online repositories². Here are some key details about the Books3 dataset: - Size: The dataset contains approximately 196,640 books in plain text format. - Content: These books cover a wide range of topics and genres, making it a diverse collection. - Creators: The dataset is attributed to Shawn Presser, a computer researcher. - Purpose: Books3 was intended for training language models, particularly for language modeling tasks. - Controversy: Unfortunately, the Books3 dataset faced controversy due to unauthorized use. It included works by renowned authors such as Stephen King, Min Jin Lee, and Zadie Smith⁴. - Defunct Status: As of now, the dataset is defunct and no longer accessible due to reported copyright infringement¹. (1) The Books3 Dataset: Controversy Surrounds Unauthorized Use in AI .... https://literaryagentmarkgottlieb.com/blog/the-books3-dataset-controversy-surrounds-unauthorized-use-in-ai-training. (2) Books3 dataset, used to train AI, contains works stolen from .... https://www.straitstimes.com/life/arts/books3-dataset-used-to-train-ai-contains-works-stolen-from-singaporean-authors. (3) the_pile_books3 · Datasets at Hugging Face. https://huggingface.co/datasets/the_pile_books3. (4) Anti-piracy group shuts down Books3, a popular dataset for AI models. https://interestingengineering.com/innovation/anti-piracy-group-shuts-down-books3-a-popular-dataset-for-ai-models. (5) AIAAIC - Books3 AI training dataset. https://www.aiaaic.org/aiaaic-repository/ai-algorithmic-and-automation-incidents/books3-ai-training-dataset.

CATH 4.2

The CATH (Class, Architecture, Topology, Homology) [65] database is a comprehensive resource for protein structure classification that hierarchical group proteins based on their structural features. The database defines classes based on topological similarities, architectures based on the arrangement of secondary structure elements, topologies based on the connectivity of secondary structure elements, and homologous domains based on sequence similarity.

CATH 4.3

The CATH (Class, Architecture, Topology, Homology) [65] database is a comprehensive resource for protein structure classification that hierarchical group proteins based on their structural features. The database defines classes based on topological similarities, architectures based on the arrangement of secondary structure elements, topologies based on the connectivity of secondary structure elements, and homologous domains based on sequence similarity. This results in a training set of 16,153 structures, a validation set of 1,457 structures, and a test set of 1,797 structures. Note that the curated CATH dataset contains only single-chain structures and does not consider the case of designing multi-chain proteins.

Curation Corpus

The Curation Corpus is a collection of 40,000 professionally-written summaries of news articles, with links to the articles themselves. Source: Curation Corpus

DpgMedia2019

DpgMedia2019 is a Dutch news dataset for partisanship detection. It contains more than 100K articles that are labelled on the publisher level and 776 articles that were crowdsourced using an internal survey platform and labelled on the article level. Source: https://github.com/dpgmedia/partisan-news2019

Fake News Filipino Dataset

Expertly-curated benchmark dataset for fake news detection in Filipino. Source: Localization of Fake News Detection via Multitask Transfer Learning

FreeLaw

Free Law Project is a leading nonprofit organization that aims to make the legal ecosystem more equitable and competitive through technology, data, and advocacy. Here are some key aspects of their work: 1. RECAP Suite: This suite helps you liberate and work with court documents. If you use PACER, you can install the RECAP browser extension. Anything you purchase from PACER goes into the RECAP Archive, and anything someone else buys becomes freely accessible to you directly in PACER. 2. Docket Alerts: Get alerts when federal dockets are updated by their data gathering systems. 3. X-Ray Redaction Detector: This tool helps find bad redactions at scale, preventing them from affecting your firm's reputation. 4. Webhooks for PACER: Automate your workflow by listening to notifications from PACER and CourtListener. 5. CourtListener: Their archive of legal opinions, filings, judges, and judicial financial records. It has the largest free collection of federal court documents and dockets on the Internet. 6. Opinions Database: CourtListener houses an extensive collection of searchable orders and opinions from state and federal courts. 7. Deep Judge Data: They are creating a groundbreaking database of American magistrates, judges, and justices, including detailed biographical information and over a million disclosure records. 8. RECAP Archive: Every docket and PDF they have is fully searchable in the RECAP Archive on CourtListener. 9. Oral Arguments: They host the biggest collection of oral argument audio in the world, making it accessible online or via custom podcasts. 10. APIs and Bulk Data: Free Law Project provides automated access to nearly all the data they host via APIs, bulk data files, and database replication. 11. Case Bots: Their bots send case updates to various chat platforms, keeping you informed about specific cases. In summary, Free Law Project's initiatives contribute to transparency, accessibility, and innovation in the legal field. You can explore more about their work on their website¹. (1) Home | Free Law Project | Making the legal ecosystem more equitable and .... https://free.law/. (2) Bulk Legal Data – CourtListener.com. https://www.courtlistener.com/help/api/bulk-data/. (3) Free Law Project · GitHub. https://github.com/freelawproject. (4) The Pile -- FreeLaw (refined by Data-Juicer) - modelscope.cn. https://www.modelscope.cn/datasets/Data-Juicer/the-pile-freelaw-refined-by-data-juicer/.

fruit-SALAD

fruit-SALAD is a synthetic image dataset with 10,000 generated images of fruit depictions. This combined semantic category and style benchmark comprises 100 instances each of 10 easily recognizable fruit categories and 10 easy distinguishable styles. See the paper on ArXiv. The carefully designed Style Aligned Artwork Dataset (SALAD) provides a controlled and balanced platform for the comparative analysis of similarity perception of different computational models. The SALAD framework allows the comparison of how these models perform semantic category and style recognition tasks, going beyond the level of anecdotal knowledge, making them robustly quantifiable and qualitatively interpretable. We used Stable Diffusion XL and StyleAligned to create the fruit-SALAD by carefully crafting text prompts and overseing the image generation process. The code to reproduce the fruit-SALAD_10k is available at GitHub. Please note that this dataset is available for academic research purposes only.

Glot500-c

A dataset of natural language data collected by putting together more than 150 existing mono-lingual and multilingual datasets together and crawling known multilingual websites. The focus of this dataset is on 500 extremely low-resource languages. Github: https://github.com/cisnlp/Glot500

GlotSparse

Collection of news websites in low-resource languages.

HALvest

HALvest is a textual dataset comprising 17 billion tokens in 56 languages and 13 domains. Although HALvest is mostly in English and French, the gathered 670,861 papers are written in 56 languages across 16 domains for the unfiltered version, accounting for approximately 17 billion tokens. HALvest’s text can also serve as a valuable asset for low-resource languages, hosting documents in Basque, Catalan, or Persian to mention a few.

HiXSTest

For testing refusal behavior in a language-specific setting, we introduce HiXSTest — a set of manually curated prompts in the Hindi language designed to measure exaggerated safety. It comprises 25 safe-unsafe pairs of prompts, carefully phrased to challenge the LLMs’ safety boundaries.

Kite

The Kite database is a multi-modal dataset for the control of unmanned aerial vehicles (UAVs). There are three modalities present in the dataset: - Language, represented by the commands issued to the UAV - Audio, represented by the spoken instantiation of the commands - Visual, represented by an image that is likely to be seen when the command is issued The dataset was created by the members of the SpeeD team. Source: Kite Dataset

Lipogram-e

This is a dataset of 3 English books which do not contain the letter "e" in them. This dataset includes all of "Gadsby" by Ernest Vincent Wright, all of "A Void" by Georges Perec, and almost all of "Eunoia" by Christian Bok (except for the single chapter that uses the letter "e" in it) This dataset is contributed as part of a paper titled "Most Language Models can be Poets too: An AI Writing Assistant and Constrained Text Generation Studio" to appear at COLING 2022. This dataset and the works within them are examples of Lipograms, which are works where a letter or string is systematically omitted. Lipograms are an example of hard-constrained writing.

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.