Papers with Code 아카이브

Text Classification

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
16
평가표
101
보존된 평가 기록
341

Text Classification is the task of assigning a sentence or document an appropriate category. The categories depend on the chosen dataset and can range from topics.

Text Classification problems include emotion classification, news classification, citation intent classification, among others. Benchmark datasets for evaluating text classification capabilities include GLUE, AGNews, among others.

In recent years, deep learning techniques like XLNet and RoBERTa have attained some of the biggest performance jumps for text classification problems.

<span style="color:grey; opacity: 0.6">( Image credit: Text Classification Algorithms: A Survey )</span>

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

CSL (Chinese Scientific Literature)

We present CSL, a large-scale Chinese Scientific Literature dataset, which contains the titles, abstracts, keywords and academic fields of 396,209 papers. To our knowledge, CSL is the first scientific document dataset in Chinese. Paper | Code and data ## Dataset We obtain the paper's meta-information from the National Engineering Research Center for Science and Technology Resources Sharing Service (NSTR) dated from 2010 to 2020. Then, we filter data by the Catalogue of Chinese Core Journals. According to the Catalogue and collected data, we divide academic fields into 13 first-level categories (e.g., Engineering, Science) and 67 second-level disciplines (e.g., Mechanics, Mathematics). In total, we collect 396,209 instances for the CSL dataset, represented as tuples <T, A, K, c, d>, where T is the title, A is the abstract, K is a list of keywords, c is the category label and d is the discipline label. The paper distribution over categories and the examples of disciplines are shown in below: | Category | #d | len(T) | len(A) | num(K) | #Samples | Discipline Examples | |-----------------|-------------:|-------:|-------:|-------:|----------:|---------------------------------------| | Engineering | 27 | 19.1 | 210.9 | 4.4 | 177,600 | Mechanics,Architecture,Electrical Science | | Science | 9 | 20.7 | 254.4 | 4.3 | 35,766 | Mathematics,Physics,Astronomy,Geography | | Agriculture | 7 | 17.1 | 177.1 | 7.1 | 39,560 | Crop Science,Horticulture,Forestry | | Medicine | 5 | 20.7 | 269.5 | 4.7 | 36,783 | Clinical Medicine,Dental Medicine,Pharmacy | | Management | 4 | 18.7 | 157.7 | 6.2 | 23,630 | Business Management,Public Administration | | Jurisprudence | 4 | 18.9 | 174.4 | 6.1 | 21,554 | Legal Science,Political Science,Sociology | | Pedagogy | 3 | 17.7 | 179.4 | 4.3 | 16,720 | Pedagogy,Psychology,Physical Education | | Economics | 2 | 19.5 | 177.2 | 4.5 | 11,558 | Theoretical Economics,Applied Economics | | Literature | 2 | 18.8 | 158.2 | 8.3 | 10,501 | Chinese Literature,Journalism | | Art | 1 | 17.8 | 170.8 | 5.4 | 5,201 | Art | | History | 1 | 17.6 | 181.0 | 6.0 | 6,270 | History | | Strategics | 1 | 17.5 | 169.3 | 4.0 | 3,555 | Military Science | | Philosophy | 1 | 18.0 | 176.5 | 8.0 | 7,511 | Philosophy | | All | 67 | | | | 396,209 | ## Evaluation Tasks We build a benchmark to facilitate the development of Chinese scientific literature NLP. It contains diverse tasks, ranging from classification to text generation, representing many practical scenarios. We randomly select 100k samples and split the datasets into the training set, validation set and test set according to the ratio, 0.8 : 0.1 : 0.1. This split is shared across different tasks, which allows multitask training and evaluation. Datasets are presented in text2text format. #### 1.Text Summarization (Title Prediction) Predict the paper title from the abstract. Data examples: { "prompt": "to title", "text_a": "多个相邻场景同时进行干涉参数外定标的过程称为联合定标,联合定标能够 \ 保证相邻场景的高程衔接性,能够实现无控制点场景的干涉定标.该文提出了 \ 一种适用于机载InSAR系统的联合定标算法...", "text_b": "基于加权最优化模型的机载InSAR联合定标算法" } #### 2.Keyword Generation Predict a list of keywords from a given paper title and abstract. Data examples: { "prompt": "to keywords", "text_a": "通过对72个圆心角为120°的双跨偏心支承弯箱梁桥模型的计算分析,以梁 \ 格系法为基础编制的3D-BSA软件系统为结构计算工具,用统计分析的方法建 \ 立双跨偏心支承弯箱梁桥结构反应在使用极限状态及承载能力极限状态下与 \ 桥梁跨长... 偏心支承对120°圆心角双跨弯箱梁桥的影响", "text_b": "曲线桥_箱形梁_偏心支承_设计_经验公式" } #### 3.Category Classification Predict the category with the paper title (13 classes). Data examples: { "prompt": "to category", "text_a": "基于模糊C均值聚类的流动单元划分方法——以克拉玛依油田五3中区克下组为例", "text_b": "工学" }, { "prompt": "to category", "text_a": "正畸牵引联合牙槽外科矫治上颌尖牙埋伏阻生的临床观察", "text_b": "医学" } #### 4.Discipline Classification Predict the discipline with the paper abstract (67 classes). Data examples: { "prompt": "to discipline", "text_a": "某铁矿选矿厂所产铁精矿含硫超过0.3%,而现场为了今后发展的需要,要 \ 求将含硫量降到0.1%以下.为此,针对该铁精矿中硫化物主要以磁黄铁矿 \ 形式存在、硫化物多与铁矿物连生且氧化程度较高的特点...", "text_b": "矿业工程" }, { "prompt": "to discipline", "text_a": "为了校正广角镜头的桶形畸变,提出一种新的桶形畸变数字校正方法.它 \ 使用点阵样板校正的方法,根据畸变图和理想图中圆点的位置关系,得出 \ 畸变图像素在X轴和Y轴方向上的偏移量曲面...", "text_b": "计算机科学与技术" }

Dataset of Propaganda Techniques of the State-Sponsored Information Operation of the People's Republic of China

This data is for the Mis2-KDD 2021 under review paper: Dataset of Propaganda Techniques of the State-Sponsored Information Operation of the People’s Republic of China We present our dataset that focuses on propaganda techniques in Mandarin based on a state-linked information operations dataset from the PRC released by Twitter in July 2019. The dataset consists of multi-label propaganda techniques of the sampled tweets. In total, we have 9,950 labeled tweets with 21 different propaganda techniques. The tweets are the state-linked information operations dataset from the PRC released by Twitter.

DEJAN Synthetic Taxonomy Dataset

This dataset contains synthetic text data generated to train models for text generation. The data is created using a large language model (LLM), specifically the GEMMA-2-9B-IT model, and is designed to encompass a diverse range of scenarios and writing styles.

E2E Refined

E2E Refined is a dataset for sentence classification. It consists of 40,560 examples for training, 4,489 for validation, and 4,555 for test. It is a refined version of the well-known MR-to-text E2E dataset where many deletion/insertion/substitution errors has been fixed.

FairFD

A Racial Fairness Benchmark Dataset for Face Forgery Detection.

FedNLP

We collect the various forms of Federal Reserve communications. - fomc_doc: FOMC post-meeting statements, Minutes, and press conferences. (122 docs) - speaker_doc: FOMC members’ speeches from over 30 different websites. (1300 docs) - period: Jan 2015 - Jul 2020 Open In Colab

FinSen

Enhancing Financial Market Predictions: Causality-Driven Feature Selection This paper introduces FinSen dataset that revolutionizes financial market analysis by integrating economic and financial news articles from 197 countries with stock market data. The dataset’s extensive coverage spans 15 years from 2007 to 2023 with temporal information, offering a rich, global perspective 160,000 records on financial market news. Our study leverages causally validated sentiment scores and LSTM models to enhance market forecast accuracy and reliability. # Our FinSen Dataset arXiv Pytorch 1.5 License: MIT This repository contains the dataset for Enhancing Financial Market Predictions: Causality-Driven Feature Selection, which has been accepted in ADMA 2024. If the dataset or the paper has been useful in your research, please add a citation to our work: @article{liang2024enhancing, title={Enhancing Financial Market Predictions: Causality-Driven Feature Selection}, author={Liang, Wenhao and Li, Zhengyang and Chen, Weitong}, journal={arXiv e-prints}, pages={arXiv--2408}, year={2024} } ### Datasets [FinSen] can be downloaded manually from the repository as csv file. Sentiment and its score are generated by FinBert model from the Hugging Face Transformers library under the identifier "ProsusAI/finbert". (Araci, Dogu. "Finbert: Financial sentiment analysis with pre-trained language models." arXiv preprint arXiv:1908.10063 (2019).) We only provide US for research purpose usage, please contact w.liang@adelaide.edu.au for other countries (total 197 included) if necessary. We also provide other NLP datasets for text classification tasks here, please cite them correspondingly once you used them in your research if any. 1. 20Newsgroups. Joachims, T., et al.: A probabilistic analysis of the rocchio algorithm with tfidf for text categorization. In: ICML. vol. 97, pp. 143–151. Citeseer (1997) 2. AG News. Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. Advances in neural information processing systems 28 (2015) 3. Financial PhraseBank. Malo, P., Sinha, A., Korhonen, P., Wallenius, J., Takala, P.: Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology 65(4), 782–796 (2014) ### Dataloader for FinSen We provide the preprocessing file finsen.py for our FinSen dataset under dataloaders directory for more convienient usage. ### Models - Text Classification 1. DAN-3. 2. Gobal Pooling CNN. ### Models - Regression Prediction 1. LSTM ### Using Sentiment Score from FinSen Predict Result on S&P500 ### Dependencies The code is based on PyTorch under code frame of https://github.com/torrvision/focal_calibration, please cite their work if you found it is useful. :smiley: ☺ Happy Research !

FMC-MWO2KG

The Failure Mode Classification dataset released in the paper "MWO2KG and Echidna: Constructing and exploring knowledge graphs from maintenance data" by Stewart et al. The goal is to label a given observation (made by a maintainer) with the corresponding Failure Mode Code. Each row contains an observation made by a maintainer, followed by a comma, followed by the Failure Mode, for example: falure,Breakdown As they are written in technical language, there are often spelling/grammatical/tokenisation errors made in the observations - these are typical of maintenance work orders. The dataset comprises 502 (observation, label) pairs (for training), 62 pairs (for validation) and 62 pairs (for testing). The labels are taken from a set of 22 failure mode codes from ISO 14224. In order to pull a list of observations in which to label, we ran MWO2KG over the data once and exported a list of all entities labelled as ‘observation’ (such as ‘leaking’, ‘not working’) by the Named Entity Recognition model. We then removed all results that were incorrectly predicted as observations by the NER model and proceeded to label each observation with the most appropriate failure mode code using a text editor. The source code of the above paper (which also includes this dataset) is located on GitHub. The direct link to the data (train.txt, dev.txt, and test.txt) is available here.

Forex News Annotated Dataset for Sentiment Analysis

This dataset contains news headlines relevant to key forex pairs: AUDUSD, EURCHF, EURUSD, GBPUSD, and USDJPY. The data was extracted from reputable platforms Forex Live and FXstreet over a period of 86 days, from January to May 2023. The dataset comprises 2,291 unique news headlines. Each headline includes an associated forex pair, timestamp, source, author, URL, and the corresponding article text. Data was collected using web scraping techniques executed via a custom service on a virtual machine. This service periodically retrieves the latest news for a specified forex pair (ticker) from each platform, parsing all available information. The collected data is then processed to extract details such as the article's timestamp, author, and URL. The URL is further used to retrieve the full text of each article. This data acquisition process repeats approximately every 15 minutes. To ensure the reliability of the dataset, we manually annotated each headline for sentiment. Instead of solely focusing on the textual content, we ascertained sentiment based on the potential short-term impact of the headline on its corresponding forex pair. This method recognizes the currency market's acute sensitivity to economic news, which significantly influences many trading strategies. As such, this dataset could serve as an invaluable resource for fine-tuning sentiment analysis models in the financial realm. We used three categories for annotation: 'positive', 'negative', and 'neutral', which correspond to bullish, bearish, and hold sentiments, respectively, for the forex pair linked to each headline. The following Table provides examples of annotated headlines along with brief explanations of the assigned sentiment. Examples of Annotated Headlines Forex Pair Headline Sentiment Explanation GBPUSD Diminishing bets for a move to 12400 Neutral Lack of strong sentiment in either direction GBPUSD No reasons to dislike Cable in the very near term as long as the Dollar momentum remains soft Positive Positive sentiment towards GBPUSD (Cable) in the near term GBPUSD When are the UK jobs and how could they affect GBPUSD Neutral Poses a question and does not express a clear sentiment JPYUSD Appropriate to continue monetary easing to achieve 2% inflation target with wage growth Positive Monetary easing from Bank of Japan (BoJ) could lead to a weaker JPY in the short term due to increased money supply USDJPY Dollar rebounds despite US data. Yen gains amid lower yields Neutral Since both the USD and JPY are gaining, the effects on the USDJPY forex pair might offset each other USDJPY USDJPY to reach 124 by Q4 as the likelihood of a BoJ policy shift should accelerate Yen gains Negative USDJPY is expected to reach a lower value, with the USD losing value against the JPY AUDUSD RBA Governor Lowe’s Testimony High inflation is damaging and corrosive Positive Reserve Bank of Australia (RBA) expresses concerns about inflation. Typically, central banks combat high inflation with higher interest rates, which could strengthen AUD. Moreover, the dataset includes two columns with the predicted sentiment class and score as predicted by the FinBERT model. Specifically, the FinBERT model outputs a set of probabilities for each sentiment class (positive, negative, and neutral), representing the model's confidence in associating the input headline with each sentiment category. These probabilities are used to determine the predicted class and a sentiment score for each headline. The sentiment score is computed by subtracting the negative class probability from the positive one.

FTR-18

FTR-18 is a multilingual rumour dataset on football transfer news. Transfer rumours are continuously published by sports media. They can both harm the image of player or a club or increase the player's market value. The proposed dataset includes transfer articles written in English, Spanish and Portuguese. It also comprises Twitter reactions related to the transfer rumours. FTR-18 is suited for rumour classification tasks and allows the research on the linguistic patterns used in sports journalism.

Genocide Transcript Corpus (GTC): Topic-Based Paragraph Classification in Genocide-Related Court Transcripts

The Topic-Based Paragraph Classification in Genocide-Related Court Transcripts (GTC) dataset is the first reference corpus annotated with samples from genocide tribunals in different international criminal courts. It is made up of witness statements about violence experienced. The material consists of 1475 text passages with about 40 to 120 pages per transcript, covering 3 tribunals: the Extraordinary Chambers in the Courts of Cambodia (ECCC) - 438 pages, the International Criminal Tribunal for Rwanda (ICTR) - 566 pages, and the International Criminal Tribunal of the Former Yugoslavia (ICTY) - 416 pages. As no datasets of any kind containing genocide court transcripts have been published nor other forms of pre-structured or annotated text data in this field of research exist, the aim was to address this gap by providing a systematically annotated dataset. Potential use cases include genocide-related inquiry conducted by those who need better to access, explore, and search through extensive documentation on these topics including researchers, lawyers and other practitioners. Broadly, its stated aim is to serve 3 purposes: * (1) to provide a first reference corpus for the community * (2) to establish benchmark performances (using state-of-the-art transformer-based approaches) for the new classification task of paragraph identification of violence-related witness statements * (3) to explore first steps towards transfer learning within the domain

Human-ChatGPT texts

A dataset including texts by humans (labeled 0) and then rephrased by ChatGPT (labeled 1), created to train models for machine-generated text detection. It is a robust dataset - includes text of various lengths, and the human texts are taken from multiple sources.

iLur News Texts

iLur News Texts is a dataset of over 12000 news articles from iLur.am, categorized into 7 classes: sport, politics, weather, economy, accidents, art, society. The articles are split into train (2242k tokens) and test sets (425k tokens). Source: iLur News Texts

Invisible Mobile Keyboard Dataset

Invisible Mobile Keyboard Dataset contains user initial, age, type of mobile devices, size of the screen, time taken for typing each phrase, and annotation of typed phrases with coordinate values of the typed position (x and y points). The collected dataset is the first and only dataset for a novel IMK decoding task.

K-MHaS: Korean Multi-label Hate Speech Dataset

Korean Multi-label Hate Speech Dataset We introduce K-MHaS, a new multi-label dataset for hate speech detection that effectively handles Korean language patterns. * consisting of 109,692 utterances from Korean online news comments, labeled with 8 fine-grained hate speech classes. * data collection period: between January 2018 and June 2020. * providing (a) binary classification and (b) multi-label classification from 1(one) to 4(four) labels. * (a) binary classification: Hate Speech or Not Hate Speech * (b) fine-grained classification: Politics, Origin, Physical, Age, Gender, Religion, Race, and Profanity. For the fine-grained classification, a Hate Speech class from the binary classification is broken down into eight classes, associated with the hate speech category.

Korean UnSmile Dataset

1.9K Korean Online Hate Speech Comments for Multilabel Classification (Annotated by Three Independent Labelers per Data)

Legal Advice Reddit

Dataset Summary New dataset introduced in Parameter-Efficient Legal Domain Adaptation (Li et al., 2022) from the Legal Advice Reddit community (known as "/r/legaldvice"), sourcing the Reddit posts from the Pushshift Reddit dataset. The dataset maps the text and title of each legal question posted into one of eleven classes, based on the original Reddit post's "flair" (i.e., tag). Questions are typically informal and use non-legal-specific language. Per the Legal Advice Reddit rules, posts must be about actual personal circumstances or situations. We limit the number of labels to the top eleven classes and remove the other samples from the dataset. ## Citation Information @inproceedings{li-etal-2022-parameter, title = "Parameter-Efficient Legal Domain Adaptation", author = "Li, Jonathan and Bhambhoria, Rohan and Zhu, Xiaodan", booktitle = "Proceedings of the Natural Legal Language Processing Workshop 2022", month = dec, year = "2022", address = "Abu Dhabi, United Arab Emirates (Hybrid)", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2022.nllp-1.10", pages = "119--129", }

Lot-insts

LoT-insts contains over 25k classes whose frequencies are naturally long-tail distributed. Its test set from four different subsets: many-, medium-, and few-shot sets, as well as a zero-shot open set. To our best knowledge, this is the first natural language dataset that focuses on this long-tailed and open classification problem.

M-Phasis

A corpus of 9k German and French user comments collected from migration-related news articles. It goes beyond the hate-neutral dichotomy and is instead annotated with 23 features, which in combination become descriptors of various types of speech, ranging from critical comments to implicit and explicit expressions of hate. The annotations are performed by 4 native speakers per language and achieve high (0.77) inter-annotator agreements.

MATHWELL Human Annotation Dataset

The MATHWELL Human Annotation Dataset contains 5,084 synthetic word problems and answers generated by MATHWELL, a reference-free educational grade school math word problem generator released in MATHWELL: Generating Educational Math Word Problems Using Teacher Annotations, and comparison models (GPT-4, GPT-3.5, Llama-2, MAmmoTH, and LLEMMA) with expert human annotations for solvability, accuracy, appropriateness, and meets all criteria (MaC). Solvability means the problem is mathematically possible to solve, accuracy means the Program of Thought (PoT) solution arrives at the correct answer, appropriateness means that the mathematical topic is familiar to a grade school student and the question's context is appropriate for a young learner, and MaC denotes questions which are labeled as solvable, accurate, and appropriate. Null values for accuracy and appropriateness indicate a question labeled as unsolvable, which means it cannot have an accurate solution and is automatically inappropriate. Based on our annotations, 82.2% of the question/answer pairs are solvable, 87.3% have accurate solutions, 78.1% are appropriate, and 58.4% meet all criteria. This dataset is designed to train text classifiers to automatically label word problem generator outputs for solvability, accuracy, and appropriateness. More details about the dataset can be found in our paper.

MeSHup

Contains 1,342,667 full text articles in English, together with the associated MeSH labels and metadata, authors, and publication venues that are collected from the MEDLINE database.

MFA

The MFA (Many Faces of Anger) dataset includes 200 in-the-wild videos from North American and Persian cultures with fine-grained labels of: 'annoyed', 'anger', 'disgust', 'hatred' and 'furious' and 13 related emojis. Image source: https://arxiv.org/pdf/2112.05267.pdf

MIE Articles Dataset (1996-2024)

This dataset contains 4606 articles from 1996 to 2024 that were presented in MIE (Medical Informatics Europe Conference) conferences. This data was extracted from PubMed and topic extraction and affiliation parsing were done on it.

Modern Hebrew Sentiment Dataset

Modern Hebrew Sentiment Dataset is a sentiment analysis benchmark for Hebrew, based on 12K social media comments, and provide two instances of these data: in token-based and morpheme-based settings. Source: Representations and Architectures in Neural Sentiment Analysis for Morphologically Rich Languages: A Case Study from Modern Hebrew

Multilingual Terms of Service

The first annotated corpus for multilingual analysis of potentially unfair clauses in online Terms of Service. The data set comprises a total of 100 contracts, obtained from 25 documents annotated in four different languages: English, German, Italian, and Polish. For each contract, potentially unfair clauses for the consumer are annotated, for nine different unfairness categories.

NatCat

A general purpose text categorization dataset (NatCat) from three online resources: Wikipedia, Reddit, and Stack Exchange. These datasets consist of document-category pairs derived from manual curation that occurs naturally by their communities. Source: Natcat: Weakly Supervised Text Classification with Naturally Annotated Datasets

NOVIC Caption-Object Data

This corpus contains data files that were generated as part of the NOVIC paper (see above). This includes the complete Object Noun Dictionary, the exact templates used for the multiset prompt templating strategy, and a large dataset of 1.8M LLM-generated and templated captions assorted by target noun. The captions were generated based on all of the target nouns in the Object Noun Dictionary. The data is directly available at the following links: * Object Noun Dictionary (JSON) * Multiset prompt templates * LLM-generated captions dataset Refer to the NOVIC code and Object Noun Dictionary code for examples of how the data can be used, as well as regenerated.

OAGT

OAGL is a paper topic dataset consisting of 6942930 records which comprise various scientific publication attributes like abstracts, titles, keywords, publication years, venues, etc. The last two fields of each record are the topic id from a taxonomy of 27 topics created from the entire collection and the 20 most significant topic words. Each dataset record (sample) is stored as a JSON line in the text file.

Paper Field

Paper Field is built from the Microsoft Academic Graph and maps paper titles to one of 7 fields of study. Each field of study - geography, politics, economics, business, sociology, medicine, and psychology - has approximately 12K training examples.

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.