Papers with Code 아카이브

Sentiment Analysis

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
12
평가표
45
보존된 평가 기록
382

Sentiment Analysis is the task of classifying the polarity of a given text. For instance, a text-based tweet can be categorized into either "positive", "negative", or "neutral". Given the text and accompanying labels, a model can be trained to predict the correct sentiment.

Sentiment Analysis techniques can be categorized into machine learning approaches, lexicon-based approaches, and even hybrid methods. Some subcategories of research in sentiment analysis include: multimodal sentiment analysis, aspect-based sentiment analysis, fine-grained opinion analysis, language specific sentiment analysis.

More recently, deep learning techniques, such as RoBERTa and T5, are used to train high-performing sentiment classifiers that are evaluated using metrics like F1, recall, and precision. To evaluate sentiment analysis systems, benchmark datasets like SST, GLUE, and IMDB movie reviews are used.

Further readings:

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

TaPaCo

TaPaCo is a freely available paraphrase corpus for 73 languages extracted from the Tatoeba database. Source: TaPaCo: A Corpus of Sentential Paraphrases for 73 Languages

SST-3

SST-5 is the Stanford Sentiment Treebank 5-way classification dataset (positive, somewhat positive, neutral, somewhat negative, negative). To create SST-3 (positive, neutral, negative), the 'somewhat positive' class was merged and treated as 'positive'. Similarly, the 'somewhat negative' class was merged and treated as 'negative'.

HappyDB

HappyDB is a corpus of 100,000 crowdsourced happy moments. Source: HappyDB: A Corpus of 100,000 Crowdsourced Happy Moments

MFRC

Moral Foundations Reddit Corpus (MFRC) is a collection of 16,123 Reddit comments that have been curated from 12 distinct subreddits, hand-annotated by at least three trained annotators for 8 categories of moral sentiment (i.e., Care, Proportionality, Equality, Purity, Authority, Loyalty, Thin Morality, Implicit/Explicit Morality) based on the updated Moral Foundations Theory (MFT) framework. Source: The Moral Foundations Reddit Corpus Image Source: https://arxiv.org/pdf/2208.05545v2.pdf

MuSe-CaR

The MuSe-CAR database is a large, multimodal (video, audio, and text) dataset which has been gathered in-the-wild with the intention of further understanding Multimodal Sentiment Analysis in-the-wild, e.g., the emotional engagement that takes place during product reviews (i.e., automobile reviews) where a sentiment is linked to a topic or entity. The estimated age range of the professional, semi-professional (influncers), and casual reviewers is between the middle of 20s until the late 50s. Most are native English speakers from the UK or the US, while a small minority are non-native, yet fluent English speakers.

TSAC

Tunisian Sentiment Analysis Corpus (TSAC) is a Tunisian Dialect corpus of 17.000 comments from Facebook. Source: Sentiment Analysis of Tunisian Dialects: Linguistic Ressources and Experiments

ArSentD-LEV

The Arabic Sentiment Twitter Dataset for the Levantine dialect (ArSenTD-LEV) is a dataset of 4,000 tweets with the following annotations: the overall sentiment of the tweet, the target to which the sentiment was expressed, how the sentiment was expressed, and the topic of the tweet. Source: ArSentD-LEV: A Multi-Topic Corpus for Target-based Sentiment Analysis in Arabic Levantine Tweets

L3CubeMahaSent

L3CubeMahaSent is a large publicly available Marathi Sentiment Analysis dataset. It consists of marathi tweets which are manually labelled. This dataset contains a total of 18,378 tweets which are classified into three classes - Positive (1), Negative (-1) and Neutral (0). All tweets are present in their original form, without any preprocessing. Out of these, 15,864 tweets are considered for splitting them into train, test and validation datasets. This has been done to avoid class imbalance in the dataset. The remaining 2,514 tweets are also provided in a separate sheet.

MultiBooked

MultiBooked is a dataset for supervised aspect-level sentiment analysis in Basque and Catalan, both of which are under-resourced languages. Source: MultiBooked: A Corpus of Basque and Catalan Hotel Reviews Annotated for Aspect-level Sentiment Classification

PHINC

PHINC is a parallel corpus of the 13,738 code-mixed English-Hindi sentences and their corresponding translation in English. The translations of sentences are done manually by the annotators. Source: PHINC: A Parallel Hinglish Social Media Code-Mixed Corpus for Machine Translation

SubjQA

SubjQA is a question answering dataset that focuses on subjective (as opposed to factual) questions and answers. The dataset consists of roughly 10,000 questions over reviews from 6 different domains: books, movies, grocery, electronics, TripAdvisor (i.e. hotels), and restaurants. Each question is paired with a review and a span is highlighted as the answer to the question (with some questions having no answer). Moreover, both questions and answer spans are assigned a subjectivity label by annotators. Questions such as "How much does this product weigh?" is a factual question (i.e., low subjectivity), while "Is this easy to use?" is a subjective question (i.e., high subjectivity). Source: https://github.com/megagonlabs/SubjQA

DPM

Don’t Patronize Me! (DPM) is an annotated dataset with Patronizing and Condescending Language towards vulnerable communities.

JGLUE

JGLUE, Japanese General Language Understanding Evaluation, is built to measure the general NLU ability in Japanese.

TASD

Aspect-based sentiment analysis (ABSA) aims to detect the targets (which are composed by continuous words), aspects and sentiment polarities in text. Published datasets from SemEval-2015 and SemEval-2016 reveal that a sentiment polarity depends on both the target and the aspect. However, most of the existing methods consider predicting sentiment polarities from either targets or aspects but not from both, thus they easily make wrong predictions on sentiment polarities. In particular, where the target is implicit, i.e., it does not appear in the given text, the methods predicting sentiment polarities from targets do not work. To tackle these limitations in ABSA, this paper proposes a novel method for target-aspect-sentiment joint detection. It relies on a pre-trained language model and can capture the dependence on both targets and aspects for sentiment prediction. Experimental results on the SemEval-2015 and SemEval-2016 restaurant datasets show that the proposed method achieves a high performance in detecting target-aspect-sentiment triples even for the implicit target cases; moreover, it even outperforms the state-of-the-art methods for those subtasks of target-aspect-sentiment detection that they are competent to.

TUNIZI

A sentiment analysis Tunisian Arabizi Dataset, collected from social networks, preprocessed for analytical studies and annotated manually by Tunisian native speakers. Source: TUNIZI: a Tunisian Arabizi sentiment analysis Dataset

DSC (10 tasks)

A set of 10 DSC datasets (reviews of 10 products) to produce sequences of tasks. The products are Sports, Toys, Tools, Video, Pet, Musical, Movies, Garden, Offices, and Kindle. 2500 positive and 2500 negative training reviews per task . The validation reviews are with 250 positive and 250 negative and the test reviews are with 250 positive and 250 negative reviews. The detailed statistic on page https://github.com/ZixuanKe/PyContinual

PerSenT

PerSenT is a dataset of crowd-sourced annotations of the sentiment expressed by the authors towards the main entities in news articles. The dataset also includes paragraph-level sentiment annotations to provide more fine-grained supervision for the task. Source: Author's Sentiment Prediction

RuSentRel

RuSentRel is a corpus of analytical articles translated into Russian texts in the domain of international politics obtained from foreign authoritative sources. The collected articles contain both the author's opinion on the subject matter of the article and a large number of references mentioned between the participants of the described situations. In total, 73 large analytical texts were labeled with about 2000 relations. Source: RuSentRel

Amazon Fine Foods

Amazon Fine Foods is a dataset that consists of reviews of fine foods from amazon. The data span a period of more than 10 years, including all ~500,000 reviews up to October 2012. Reviews include product and user information, ratings, and a plaintext review. Source: https://snap.stanford.edu/data/web-FineFoods.html

CAIL2019-SCM

Chinese AI and Law 2019 Similar Case Matching dataset. CAIL2019-SCM contains 8,964 triplets of cases published by the Supreme People's Court of China. CAIL2019-SCM focuses on detecting similar cases, and the participants are required to check which two cases are more similar in the triplets. Source: CAIL2019-SCM: A Dataset of Similar Case Matching in Legal Domain

DiaASQ

DiaASQ is a fine-grained Aspect-based Sentiment Analysis (ABSA) benchmark under the conversation scenario. It challenges existing ABSA methods by 1) extracting quadruple of target-aspect-opinion-sentiment in a dialogue, and 2) modeling the dialogue discourse structures. The dataset is constructed by systematically crawling tweets from digital bloggers, followed by a series of preprocessing steps including filtering, normalizing, pruning, and annotating the collected dialogues, resulting in a final corpus of 1,000 dialogues. To enhance the multilingual usability, DiaASQ has both the English and Chinese versions of languages.

GeoCoV19

GeoCoV19 is a large-scale Twitter dataset containing more than 524 million multilingual tweets. The dataset contains around 378K geotagged tweets and 5.4 million tweets with Place information. The annotations include toponyms from the user location field and tweet content and resolve them to geolocations such as country, state, or city level. In this case, 297 million tweets are annotated with geolocation using the user location field and 452 million tweets using tweet content.

Laptop-ACOS

Laptop-ACOS is a brand new Laptop dataset collected from the Amazon platform in the years 2017 and 2018 (covering ten types of laptops under six brands such as ASUS, Acer, Samsung, Lenovo, MBP, MSI, and so on). It contains 4,076 review sentences, much larger than the SemEval Laptop datasets. For Laptop-ACOS, we annotate the four elements and their corresponding quadruples all by ourselves. We employ the aspect categories defined in the SemEval 2016 Laptop dataset. The Laptop-ACOS dataset contains 4076 sentences with 5758 quadruples. As we have mentioned, a large percentage of the quadruples contain implicit aspects or implicit opinions . By comparing two datasets, it can be observed that Laptop-ACOS has a higher percentage of implicit opinions than Restaurant-ACOS . It is worth noting that the Laptop-ACOS is available for all subtasks in ABSA, including aspect-based sentiment classification, aspect-sentiment pair extraction, aspect-opinion pair extraction, aspect-opinion sentiment triple extraction, aspect-category-sentiment triple extraction, etc.

MuseASTE

•A new benchmark dataset for Aspect Sentiment Triplet Extraction. •First Aspect Sentiment Triplet Extraction (ASTE) Dataset in Automotive Domain. •Largest ASTE Dataset to date with annotations for over 28,295 sentences. •Dataset includes complex aspects not verbatim present in the sentence. •Domain: Aspect-based sentiment analysis, ASTE, Opinion Mining, Recommender System. •Four baseline SOTA models implemented on the dataset

Restaurant-ACOS

The Restaurant-ACOS dataset is constructed based on the SemEval 2016 Restaurant dataset (Pontiki et al., 2016) and its expansion datasets (Fan et al., 2019; Xu et al., 2020). The SemEval 2016 Restaurant dataset (Pontiki et al., 2016) was annotated with explicit and implicit aspects, categories, and sentiment. (Fan et al., 2019; Xu et al., 2020) further added the opinion annotations. We integrate their annotations to construct aspect-category-opinion-sentiment quadruples and further annotate the implicit opinions. The Restaurant-ACOS dataset contains 2286 sentences with 3658 quadruples. It is worth noting that the Restaurant-ACOS is available for all subtasks in ABSA, including aspect-based sentiment classification, aspect-sentiment pair extraction, aspect-opinion pair extraction, aspect-opinion sentiment triple extraction, aspect-category-sentiment triple extraction, etc.

Twitter Sentiment Analysis

This is an entity-level Twitter Sentiment Analysis dataset. For each message, the task is to judge the sentiment of the entire sentence towards a given entity. For example, A outperforms B is positive for entity A but negative for entity B. The dataset contains ~70K labeled training messages and 1K labeled validation messages. It is available online for free on Kaggle.

UIT-VSMEC

Emotion recognition is a higher approach or special case of sentiment analysis. In this task, the result is not produced in terms of either polarity: positive or negative or in the form of rating (from 1 to 5) but of a more detailed level of sentiment analysis in which the result are depicted in more expressions like sadness, enjoyment, anger, disgust, fear and surprise. Emotion recognition plays a critical role in measuring brand value of a product by recognizing specific emotions of customers’ comments. In this study, we have achieved two targets. First and foremost, we built a standard Vietnamese Social Media Emotion Corpus (UIT-VSMEC) with about 6,927 human-annotated sentences with six emotion labels, contributing to emotion recognition research in Vietnamese which is a low-resource language in Natural Language Processing (NLP). Secondly, we assessed and measured machine learning and deep neural network models on our UIT-VSMEC. As a result, Convolutional Neural Network (CNN) model achieved the highest performance with 57.61% of F1-score.

XED

XED is a multilingual fine-grained emotion dataset. The dataset consists of human-annotated Finnish (25k) and English sentences (30k), as well as projected annotations for 30 additional languages, providing new resources for many low-resource languages. Source: XED: A Multilingual Dataset for Sentiment Analysis and Emotion Detection

Pars-ABSA

Pars-ABSA is a manually annotated Persian dataset, Pars-ABSA, which is verified by 3 native Persian speakers. The dataset consists of 5,114 positive, 3,061 negative and 1,827 neutral data samples from 5,602 unique reviews. Source: Pars-ABSA: an Aspect-based Sentiment Analysis dataset for Persian

RETWEET

RETWEET is a dataset of tweets and overall predominant sentiment of their replies. SUMMARY ------ WHAT: Message-level Polarity Classification. GOAL: To predict the predominant sentiment among (potential) first-order replies to a given tweet. IDEA: Mitigate the problem of lacking labeled training data wi treating the unsupervised nature of the problem as a supervised learning case. ### APPROACH: 1. Train a tweet classifier. 2. Automatically label the replies using the classifier trained in the first part. 3. Choose a final label representing the general predominant sentiment of the replies of every tweet. ### DATA COLLECTION To download all of the replies to a tweet, the Search API should be used. However, the Search API is limited to 75000 requests per hour, which causes the mining and downloading process to be slow. Furthermore, using the Twitter API, there is no possibility of downloading absolute random data. Therefore, we try to make the procedure as random as possible by utilizing two different strategies for data downloading and using them in an intermixed manner. 1. Our first strategy is based on a sample of English tweets obtained by filtering the Twitter stream via a list of cultural keywords. This list consists of 147 words that are deemed to play a "pivotal role in discussions of culture and society", covering diverse words such as aesthetics, environment, feminism, power, tourism, or youth. We extracted all tweets in 2019 that have a minimum of 20 first-order replies in the dataset. The data come with an obvious caveat: Both the source tweet as well as all the replies must contain at least one word from the list of keywords. Therewith, it is highly unlikely that the list of replies for any given source is exhaustive, i.e. there might be many more first-order replies to the source tweet that are not in the dataset. 2. As our second approach, we use the GetOldTweets3 library to download all the replies corresponding to every tweet. We define few restrictions to add randomization to the process. Firstly, every tweet and also every reply should contain at least 20 strings. This is due to the fact that our automatic tweet classifier, explsined in the paper, is optimized based on the message-level classification paradigm. Therefore, it operates optimal when the input contains at least a sufficient number of words. The second constraint is that every tweet should contain at least 20 first-order replies. In order to increase randomness, in this strategy, instead of referencing to a list of keywords, we manually choose some keywords, which are most likely to include long discussions, such as Coronavirus and football or the ones, which are most likely to include strong opinions such as birthday, war, or racism in order to account for the easy-to-guess examples. ### MANUAL ANNOTATIONS FOR THE RETWEET (TEST GOLD DATASET) 5,015 tweets with their corresponding replies, collected as a combination of the two different collection strategies, were given to three different students. Each of them had to read all the replies corresponding to every tweet, without observing the original tweet in order to avoid having a prior knowledge, and decide on ONE final sentiment for the replies. The assigned sentiment can only be one of the positive, negative, or neutral labels. Considering the fact that this is a really challenging task for the machine, to prevent human mistakes, we correlated the results of the three annotators and only chose the tweets, in which all of the annotators had the same opinion on the labels, as the final gold standard test data. Therefore, we finally, ended up with a test set consisting of 1,519 human labeled tweets, with the labels being the sentiment of the replies of a tweet and not the tweet itself. DATASET CONTENTS --- 1. Training raw dataset: 34,953 unique tweets in total and individual automatic labels for all of their corresponding replies (1,519,504 total replies). Including, - ./RETWEET_data/train_reply_labels_set1.txt - ./RETWEET_data/train_reply_labels_set2.txt 2. Training autamtically-labeled dataset: 34,953 unique tweets and ONE final automatic label (chosen based on the algorithm 1 of our paper) for every tweet. Including, - ./RETWEET_data/train_final_label.txt 3. Gold standard test dataset (RETWEET): 1,519 unique tweets with their manual labels for replies. ONE final label, which states the predominant overall polarity of all its replies, is assigned to every tweet. Including, - ./RETWEET_data/test_gold.txt NOTES --- 1. Please note that by downloading the Twitter data you agree to abide by the Twitter terms of service, and in particular you agree not to redistribute the data and to delete tweets that are marked deleted in the future. 2. The "neutral" label in the annotations stands for objective or neutral. 3. The distribution consists of a set of Twitter unique tweet IDs with annotations (overall polarity of replies). As for data privacy, the texts of the tweets and replies are not distributed. But as all the utilized resources in this dataset are taken from public tweets, having the tweet unique IDs, you can download the tweet and its replies. You can use the Semeval Twitter data downloading script to obtain the corresponding tweets: https://github.com/seirasto/twitter_download/ 4. The dataset URL: https://kaggle.com/soroosharasteh/retweet/ LICENSE --- The accompanying dataset is released under a Creative Commons Attribution 4.0 International License. SOURCE CODE --- The official source code of the paper: https://github.com/starasteh/retweet ### In case you use this dataset, please cite the original paper: S. Tayebi Arasteh, M. Monajem, V. Christlein, P. Heinrich, A. Nicolaou, H.N. Boldaji, M. Lotfinia, S. Evert. "How Will Your Tweet Be Received? Predicting the Sentiment Polarity of Tweet Replies". Proceedings of the 2021 IEEE 15th International Conference on Semantic Computing (ICSC), Laguna Hills, CA, USA, January 2021. ### BibTex @inproceedings{RETWEET, title = "How Will Your Tweet Be Received? Predicting the Sentiment Polarity of Tweet Replies", author = "Tayebi Arasteh, Soroosh and Monajem, Mehrpad and Christlein, Vincent and Heinrich, Philipp and Nicolaou, Anguelos and Naderi Boldaji, Hamidreza and Lotfinia, Mahshad and Evert, Stefan", booktitle = "Proceedings of the 2021 IEEE 15th International Conference on Semantic Computing (ICSC)", address = "Laguna Hills, CA, USA", pages = "370-373", doi = "10.1109/ICSC50631.2021.00068", url = "https://ieeexplore.ieee.org/document/9364527/", month = "01", year = "2021" } * Dataset DOI: 10.34740/kaggle/ds/736988 * Paper: https://ieeexplore.ieee.org/document/9364527 * Paper DOI: 10.1109/ICSC50631.2021.00068 CONTACT --- E-mail: soroosh.arasteh@fau.de DATA FORMAT FOR ALL THE FILES --- label TAB id where, "label" can be positive, neutral or negative, corresponding to the overall message-level polarity of the replies of the tweet and "id" corresponds to the Twitter unique ID for the tweets.

SDCNL (Suicide vs Depression Classification)

We develop a primary dataset based on our task of suicide or depression classification. This dataset is web-scraped from Reddit. We collect our data from subreddits using the Python Reddit API. We specifically scrape from two subreddits, r/SuicideWatch3 and r/Depression. The dataset contains 1,895 total posts. We utilize two fields from the scraped data: the original text of the post as our inputs, and the subreddit it belongs to as labels. Posts from r/SuicideWatch are labeled as suicidal, and posts from r/Depression are labeled as depressed. We make this dataset and the web-scraping script available in our code.

SPIRS

A first-of-its-kind large dataset of sarcastic/non-sarcastic tweets with high-quality labels and extra features: (1) sarcasm perspective labels (2) new contextual features. The dataset is expected to advance sarcasm detection research. Source: Reactive Supervision: A New Method for Collecting Sarcasm Data

SPOT

The SPOT dataset contains 197 reviews originating from the Yelp'13 and IMDB collections ([1][2]), annotated with segment-level polarity labels (positive/neutral/negative). Annotations have been gathered on 2 levels of granulatiry: - Sentences - Elementary Discourse Units (EDUs), i.e. sub-sentence clauses produced by a state-of-the-art RST parser This dataset is intended to aid sentiment analysis research and, in particular, the evaluation of methods that attempt to predict sentiment on a fine-grained, segment-level basis. Source: SPOT

TCAB

Text Classification Attack Benchmark (TCAB) is a dataset for analyzing, understanding, detecting, and labeling adversarial attacks against text classifiers. TCAB includes 1.5 million attack instances, generated by twelve adversarial attack targeting three classifiers trained on six source datasets for sentiment analysis and abuse detection in English. The process of generating attacks is automated, so that TCAB can easily be extended to incorporate new text attacks and better classifiers as they are developed. Source: TCAB: A Large-Scale Text Classification Attack Benchmark

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.