Stance Detection

The AI2’s Reasoning Challenge (ARC) dataset is a multiple-choice question-answering dataset, containing questions from science exams from grade 3 to grade 9. The dataset is split in two partitions: Easy and Challenge, where the latter partition contains the more difficult questions that require reasoning. Most of the questions have 4 answer choices, with <1% of all the questions having either 3 or 5 answer choices. ARC includes a supporting KB of 14.3M unstructured text passages. Source: Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering Image Source: https://arxiv.org/abs/1803.05457

LIAR is a publicly available dataset for fake news detection. A decade-long of 12.8K manually labeled short statements were collected in various contexts from POLITIFACT.COM, which provides detailed analysis report and links to source documents for each case. This dataset can be used for fact-checking research as well. Notably, this new dataset is an order of magnitude larger than previously largest public fake news datasets of similar type. The LIAR dataset4 includes 12.8K human labeled short statements from POLITIFACT.COM’s API, and each statement is evaluated by a POLITIFACT.COM editor for its truthfulness. Source: “Liar, Liar Pants on Fire”: A New Benchmark Dataset for Fake News Detection

P-Stance: A Large Dataset for Stance Detection in Political Domain 2021

Fact-checking (FC) articles which contains pairs (multimodal tweet and a FC-article) from snopes.com. Source: Where Are the Facts? Searching for Fact-checked Information to Alleviate the Spread of Fake News

FNC-1 was designed as a stance detection dataset and it contains 75,385 labeled headline and article pairs. The pairs are labelled as either agree, disagree, discuss, and unrelated. Each headline in the dataset is phrased as a statement Source: Investigating Rumor News Using Agreement-Aware Search Image Source: http://www.fakenewschallenge.org/

VAST consists of a large range of topics covering broad themes, such as politics (e.g., ‘a Palestinian state’), education (e.g., ‘charter schools’), and public health (e.g., ‘childhood vaccination’). In addition, the data includes a wide range of similar expressions (e.g., ‘guns on campus’ versus ‘firearms on campus’). This variation captures how humans might realistically describe the same topic and contrasts with the lack of variation in existing datasets. Source: Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations

A large-scale stance detection dataset from comments written by candidates of elections in Switzerland. The dataset consists of German, French and Italian text, allowing for a cross-lingual evaluation of stance detection. It contains 67 000 comments on more than 150 political issues (targets). Source: X-Stance: A Multilingual Multi-Target Dataset for Stance Detection

Perspectrum is a dataset of claims, perspectives and evidence, making use of online debate websites to create the initial data collection, and augmenting it using search engines in order to expand and diversify the dataset. Crowd-sourcing was used to filter out noise and ensure high-quality data. The dataset contains 1k claims, accompanied with pools of 10k and 8k perspective sentences and evidence paragraphs, respectively. Source: Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims

MGTAB is the first standardized graph-based benchmark for stance and bot detection. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. For more details, please refer to the MGTAB paper. Source: https://github.com/GraphDetec/MGTAB

For LIAR-RAW, we extended the public dataset LIAR-PLUS (Alhindi et al., 2018) with relevant raw reports, containing fine-grained claims from Politifact. LIAR-RAW is based on LIAR, where gold labels refer to Politifact. To alleviate the dependency of fact-checked reports, we extended the public LIAR dataset with additional raw reports for each claim. Besides, we put these raw reports into a single file with the format of LIAR. Provide: * A novel benchmark dataset for Explainable fake news detection.

For RAWFC, we constructed it from scratch by collecting the claims from Snopes and relevant raw reports by retrieving claim keywords. To alleviate the dependency of fact-checked reports, RAWFC was constructed by using raw reports (from scratch), where gold labels refer to Snopes. Each instance in the train/val/test set is presented as a signle file. Provide: * A novel benchmark dataset for explainable fake news detection

Contains 3,689,229 English news articles on politics, gathered from 11 United States (US) media outlets covering a broad ideological spectrum.

Conversational Stance Detection (CSD) is a dataset with annotations of stances and the structures of conversation threads. It consists of 500 conversation threads (including 500 posts and 5376 comments) from six major social media platforms in Hong Kong. Source: Improved Target-specific Stance Detection on Social Media Platforms by Delving into Conversation Threads Image Source: https://arxiv.org/pdf/2211.03061v1.pdf

CoVaxLies v1 includes 17 known Misinformation Targets (MisTs) found on Twitter about the covid-19 vaccines. Language experts annotated tweets as Relevant or Not Relevant, and then further annotated Relevant tweets with Stance towards each MisT. This collection is a first step in providing large-scale resources for misinformation detection and misinformation stance identification.

COVID-CQ is a stance data set of user-generated content on Twitter in the context of COVID-19. Source: A Stance Data Set on Polarized Conversations on Twitter about the Efficacy of Hydroxychloroquine as a Treatment for COVID-19 Image Source: https://arxiv.org/pdf/2009.01188v2.pdf

The dataset is annotated with stance towards one topic, namely, the independence of Catalonia. Source: Multilingual Stance Detection: The Catalonia Independence Corpus

CoVaxFrames includes 113 Vaccine Hesitancy Framings found on Twitter about the COVID-19 vaccines. Language experts annotated tweets as Relevant or Not Relevant, and then further annotated Relevant tweets with Stance towards each framing.

CoVaxLies v2 includes 47 Misinformation Targets (MisTs) found on Twitter about the COVID-19 vaccines. Language experts annotated tweets as Relevant or Not Relevant, and then further annotated Relevant tweets with Stance towards each MisT. This collection is a first step in providing large-scale resources for misinformation detection and misinformation stance identification.

MMVax-Stance includes 113 Vaccine Hesitancy Framings found on Twitter about the COVID-19 vaccines. Language experts annotated multimodal image-text tweets as Relevant or Not Relevant, and then further annotated Relevant tweets with Stance towards each framing.

Includes Russian tweets and news comments from multiple sources, covering multiple stories, as well as text classification approaches to stance detection as benchmarks over this data in this language. Source: Stance Prediction for Russian: Data and Analysis

The "Stance Detection in COVID-19 Tweets" dataset represents an evolution of stance detection research, tailored to address the unique and urgent challenges presented by the COVID-19 pandemic. This dataset is designed to capture public opinions, beliefs, and sentiments towards various aspects of the COVID-19 crisis, such as government policies, vaccination campaigns, public health recommendations, and the impact of the virus on daily life. It facilitates the analysis of how people's stances on these issues are expressed in social media discourse, specifically through tweets. ### Dataset Characteristics - Content: The dataset comprises tweets related to COVID-19, each annotated with the author's stance towards a specific target or topic related to the pandemic. These topics can range from public health measures to vaccines, government responses, and more. - Labels: Similar to other stance detection datasets, each tweet is labeled to indicate whether the stance towards the target is FAVOR, AGAINST, or NEUTRAL. - Targets: The targets within this dataset are diverse, reflecting the wide range of issues and controversies surrounding the COVID-19 pandemic. This could include specific vaccines, lockdown measures, mask mandates, and the performance of governmental and health organizations. - Volume and Language: While the size of the dataset and the languages included can vary depending on the specific collection and annotation effort, the primary focus is typically on English-language tweets given the global use of English on Twitter. The volume aims to be large enough to support robust machine learning and data analysis. ### Motivations and Summary The primary motivation behind creating a COVID-19-specific stance detection dataset is to understand and analyze the public discourse surrounding the pandemic, which has generated unprecedented levels of discussion on social media. Analyzing stances on COVID-19-related topics is crucial for public health officials, governments, and organizations to gauge public opinion, monitor misinformation, and adapt communication strategies accordingly. This dataset provides a snapshot of the evolving public sentiment towards various aspects of the pandemic, offering insights into how opinions shift over time in response to new information, policies, and developments. It's a valuable resource for researchers looking to apply NLP techniques to real-world issues with significant social impact. ### Potential Use Cases - Public Health Communication: Understanding public stances towards different aspects of the pandemic can help health organizations tailor their messaging to address concerns, combat misinformation, and encourage health-promoting behaviors. - Policy Analysis: Policymakers can use stance detection to assess public support or opposition to COVID-19 policies, guiding more effective and acceptable policy-making. - Social Research: Researchers interested in the sociology and psychology of pandemics can analyze stances to study how public attitudes towards science, authority, and community health measures evolve in crisis situations. - Misinformation Detection: By identifying prevailing stances and sentiment towards controversial topics, it's possible to pinpoint areas where misinformation may be spreading, enabling targeted interventions. The "Stance Detection in COVID-19 Tweets" dataset not only contributes to the field of computational linguistics by providing data for developing and refining stance detection algorithms but also serves a broader societal purpose by offering insights into public sentiment during one of the most challenging global health crises of the 21st century.

The data set contains 2500 manually-stance-labeled tweets, 1250 for each candidate (Joe Biden and Donald Trump). These tweets were sampled from the unlabeled set that our research team collected English tweets related to the 2020 US Presidential election. Through the Twitter Streaming API, the authors collected data using election-related hashtags and keywords. Between January 2020 and September 2020, over 5 million tweets were collected, not including quotes and retweets. Paper: Knowledge Enhanced Masked Language Model for Stance Detection

Will-They-Won't-They (WT-WT) is a large dataset of English tweets targeted at stance detection for the rumor verification task. The dataset is constructed based on tweets that discuss five recent merger and acquisition (M&A) operations of US companies, mainly from the healthcare sector. All the annotations are carried out by domain experts; therefore, the dataset constitutes a high-quality and reliable benchmark for future research in stance detection. Source: WT-WT

COVMis-Stance is a stance detection dataset for COVID-19 misinformation. It consists of fake news and claims related to COVID. Fake news was collected from articles fact-checking sites, and fake claims were from the WHO official Twitter. It contains 2631 tweets annotated for stance towards 111 COVID19 misinformation items.

Climate change poses critical challenges globally, disproportionately affecting low-income countries that often lack resources and linguistic representation on the international stage. Despite Bangladesh's status as one of the most vulnerable nations to climate impacts, research gaps persist in Bengali-language studies related to climate change and NLP. To address this disparity, we introduce ধরণী (Dhoroni), a novel Bengali (Bangla) climate change and environmental news dataset, comprising a 2300 annotated Bangla news articles, offering multiple perspectives such as political influence, scientific/statistical data, authenticity, stance detection, and stakeholder involvement. Furthermore, we present an in-depth exploratory analysis of Dhoroni and introduce BanglaBERT-Dhoroni family, a novel baseline family for climate stance detection in Bangla, fine-tuned on our dataset. This research contributes significantly to enhancing accessibility and analysis of climate discourse in Bengali (Bangla), addressing crucial communication and research gaps in climate-impacted regions like Bangladesh with 180 million people.

The ExaASC dataset is a dataset for Target-based Stance Detection in the Arabic Language that contains different types of targets like persons, entities and events. This corpus contains about 9500 tweets with replies and target specified in the source tweet. Each sample has at least two stance annotations provided by Exa Corporation annotators. The stance of each reply is annotated toward the target in the corresponding source tweet. Format of data is as follows: id, main (source tweet), reply, target, label of each annotator id and majority_label.

HpVaxFrames includes 64 Vaccine Hesitancy Framings found on Twitter about the HPV vaccines. Language experts annotated tweets as Relevant or Not Relevant, and then further annotated Relevant tweets with Stance towards each framing.

This dataset of medical misinformation was collected and is published by Kempelen Institute of Intelligent Technologies (KInIT). It consists of approx. 317k news articles and blog posts on medical topics published between January 1, 1998 and February 1, 2022 from a total of 207 reliable and unreliable sources. The dataset contains full-texts of the articles, their original source URL and other extracted metadata. If a source has a credibility score available (e.g., from Media Bias/Fact Check), it is also included in the form of annotation. Besides the articles, the dataset contains around 3.5k fact-checks and extracted verified medical claims with their unified veracity ratings published by fact-checking organisations such as Snopes or FullFact. Lastly and most importantly, the dataset contains 573 manually and more than 51k automatically labelled mappings between previously verified claims and the articles; mappings consist of two values: claim presence (i.e., whether a claim is contained in the given article) and article stance (i.e., whether the given article supports or rejects the claim or provides both sides of the argument). The dataset is primarily intended to be used as a training and evaluation set for machine learning methods for claim presence detection and article stance classification, but it enables a range of other misinformation related tasks, such as misinformation characterization or analyses of misinformation spreading.

Political stance in Danish. Examples represent statements by politicians and are annotated for, against, or neutral to a given topic/article.

A novel stance detection dataset covering 419 different controversial issues and their related pros and cons collected by procon.org in nonpartisan format. Source: Stance Prediction for Contemporary Issues: Data and Experiments

SemEval-2016 Task 6, titled "Stance Detection in Tweets," provides a specialized dataset for the computational linguistics and natural language processing (NLP) communities to explore and analyze users' positions towards certain targets, based solely on the content of their tweets. Stance detection aims to determine whether the author of a piece of text is in favor of, against, or neutral towards a specified target, such as a political figure, policy, or product. ### Dataset Characteristics - Content: The dataset consists of tweets, each labeled with the stance of the tweet towards a target. The targets are predefined topics or entities relevant to public debate or interest at the time of the dataset's creation. - Labels: Each tweet is annotated with one of three possible stance labels: FAVOR, AGAINST, or NEUTRAL. These labels represent the author's stance towards the target mentioned in the tweet. - Targets: A limited set of targets is included, each representing a contentious topic of discussion. Examples of targets might include political figures, legislation, or social issues. - Volume: The dataset includes thousands of tweets, providing a substantial volume for training and evaluating machine learning models. - Language: The primary language of the dataset is English, encompassing a wide range of linguistic expressions, idioms, and slang, reflective of the Twitter platform's diverse user base. ### Motivations and Summary The primary motivation behind the creation of this dataset was to foster advancements in stance detection, an area of sentiment analysis that goes beyond simple positive, negative, or neutral sentiment classification. Stance detection requires understanding not just the sentiment but the specific position taken towards a target, which can be challenging due to the subtlety and complexity of language used in social media. The dataset serves as a benchmark for developing and evaluating NLP models capable of automatically detecting stance in text, an essential task for applications such as opinion mining, political polling, and market research. By providing real-world, annotated data, the dataset allows for the exploration of how language is used to express support, opposition, or neutrality towards specific entities or topics. ### Potential Use Cases - Opinion Mining and Sentiment Analysis: Researchers and practitioners can use the dataset to improve algorithms that automatically identify public opinion trends on various issues, enabling more nuanced analysis than simple positive/negative sentiment classification. - Social Media Monitoring: Companies and organizations can leverage stance detection models trained on this dataset to monitor social media discourse around their products, policies, or brand to better understand public perception. - Political Analysis: Analysts interested in the public opinion dynamics of election campaigns or policy debates can use stance detection to gauge support or opposition levels among social media users. - Content Filtering and Recommendation Systems: By understanding the stance of users towards different topics, platforms can tailor content recommendations more effectively, promoting content that aligns with users' views or introducing them to counterpoints to encourage diverse exposure. Overall, the SemEval-2016 Task 6 dataset provides valuable resources for advancing the field of stance detection, with broad applications in analyzing and understanding public opinions expressed on social media platforms.

StEduCov, a dataset annotated for stances toward online education during the COVID-19 pandemic. StEduCov has 17,097 tweets gathered over 15 months, from March 2020 to May 2021, using Twitter API. The tweets are manually annotated into agree, disagree or neutral classes. We used a set of relevant hashtags and keywords. Specifically, we utilised a combination of hashtags, such as '#COVID 19' or '#Coronavirus' with keywords, such as 'education', 'online learning', 'distance learning' and 'remote learning'. To ensure high annotation quality, three different annotators annotated each tweet and at least one of the reviewers from three judges revised it. They were guided by some instructions, such as that in the case of disagree class, there should be a clear negative statement about online education or its impact. Also, if the tweet is negative but refers to other people (e.g. 'my children hate online learning').

Combines CoVaxFrames and HpVaxFrames into a unified dataset of 113 Vaccine Hesitancy Framings found on Twitter about the COVID-19 vaccines and 64 Vaccine Hesitancy Framings found on Twitter about the HPV vaccines. Language experts annotated tweets as Relevant or Not Relevant, and then further annotated Relevant tweets with Stance towards each framing.

A Natural Language Resource for Learning to Recognize Misinformation about the COVID-19 and HPV Vaccines.

This is a stance detection dataset in the Zulu language. The data is translated to Zulu by Zulu native speakers, from English source texts. Our paper aims at utilizing this progress made for English to transfers that knowledge into other languages, which is a non-trivial task due to the domain gap between English and the target languages. We propose a black-box non-intrusive method that utilizes techniques from Domain Adaptation to reduce the domain gap, without requiring any human expertise in the target language, by leveraging low-quality data in both a supervised and unsupervised manner. This allows us to rapidly achieve similar results for stance detection for the Zulu language, the target language in this work, as are found for English. A natively-translated dataset is used for evaluation of domain transfer.