Papers with Code 아카이브

Abuse Detection

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
1
평가표
0
보존된 평가 기록
0

Abuse detection is the task of identifying abusive behaviors, such as hate speech, offensive language, sexism and racism, in utterances from social media platforms (Source: https://arxiv.org/abs/1802.00385).

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

HS-BAN

HS-BAN is a binary class hate speech (HS) dataset in Bangla language consisting of more than 50,000 labeled comments, including 40.17% hate and rest are non hate speech.

K-MHaS: Korean Multi-label Hate Speech Dataset

Korean Multi-label Hate Speech Dataset We introduce K-MHaS, a new multi-label dataset for hate speech detection that effectively handles Korean language patterns. * consisting of 109,692 utterances from Korean online news comments, labeled with 8 fine-grained hate speech classes. * data collection period: between January 2018 and June 2020. * providing (a) binary classification and (b) multi-label classification from 1(one) to 4(four) labels. * (a) binary classification: Hate Speech or Not Hate Speech * (b) fine-grained classification: Politics, Origin, Physical, Age, Gender, Religion, Race, and Profanity. For the fine-grained classification, a Hate Speech class from the binary classification is broken down into eight classes, associated with the hate speech category.

KnowledJe

We introduce KnowledJe, an English-language knowledge graph of antisemitic history and language from the 20th century to the present. Structured as a JSON file, KnowledJe currently contains 618 entries, which consist of 210 event names, 137 place names, 95 person names, 80 dates (years), 38 publication names, 27 organization names, and 1 product name. Each entry is associated with its own dictionary, which contains descriptions, locations, authors, and dates as applicable. We obtain the entries through four Wikipedia articles: “Timeline of antisemitism in the 20th century,” “Timeline of antisemitism in the 21stcentury,” the “Jews” section of “List of religious slurs,” and “Timeline of the Holocaust.” To obtain descriptions for each applicable key, we used the following general rules: 1. If the concept associated with the key is a slur, the description is the entry in the “Meaning, origin, and notes” column of the “List of religious slurs” article. 2. Otherwise, if the concept associated with the key has its own Wikipedia page and that Wikipedia page has a table of contents, the description is the body of text above the table of contents. If the page exists but does not have a table of contents, the description is the first paragraph of the text on the page. 3. Otherwise, the description is the paragraph given directly under the listing of the year of the event in the Wikipedia article in which the concept was first found. We edit descriptions to remove non-Latin characters and citations. For concepts with multiple names, we create separate keys for each name. Potential use cases: enhancing hate speech detection algorithms for antisemitism, extracting knowledge of historical antisemitism. See paper (https://arxiv.org/abs/2304.11223) for examples and further description. See GitHub repo (https://github.com/enscma2/knowledje) for data files.

Korean Hate Speech Evaluation Datasets

APEACH is the first crowd-generated Korean evaluation dataset for hate speech detection. Sentences of the dataset are created by anonymous participants using an online crowdsourcing platform DeepNatural AI.

L3Cube-MahaCorpus

L3Cube-MahaCorpus is a Marathi monolingual data set scraped from different internet sources. We expand the existing Marathi monolingual corpus with 24.8M sentences and 289M tokens. We also present, MahaBERT, MahaAlBERT, and MahaRoBerta all BERT-based masked language models, and MahaFT, the fast text word embeddings both trained on full Marathi corpus with 752M tokens.

M-Phasis

A corpus of 9k German and French user comments collected from migration-related news articles. It goes beyond the hate-neutral dichotomy and is instead annotated with 23 features, which in combination become descriptors of various types of speech, ranging from critical comments to implicit and explicit expressions of hate. The annotations are performed by 4 native speakers per language and achieve high (0.77) inter-annotator agreements.

Peer to Peer Hate

Peer to Peer Hate is a comprehensive hate speech dataset capturing various types of hate. It has been built from 27,330 hate speech tweets.

SHAJ

This is an abusive/offensive language detection dataset for Albanian. The data is formatted following the OffensEval convention. Data is from Instagram and YouTube comments.

Spanish Corpus XIX

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

STATE ToxiCN

With the rise of social media, user-generated content has surged, and hate speech has proliferated. Hate speech targets groups or individuals based on race, religion, gender, region, sexual orientation, or physical traits, expressing malice or inciting harm. Recognized as a growing social issue, it affects 9.41 billion Mandarin Chinese speakers (12% of the global population). However, research on Chinese hate speech detection lags, facing two key challenges. First, existing studies focus on post-level analysis, but post-level classification cannot fully assess models' understanding of hate semantics. Hate speech intensity and directionality depend on associated targets and arguments. In Chinese, flexible word order and lack of segmentation markers (e.g., spaces in English) make fragment-level detection challenging. To address this, we created the first fragment-level Chinese hate speech dataset, annotated with (Target-Argument-Hate Group-Hateful) quadruples. Second, while priorr work provided hate word lists, the lack of interpretive annotations hinders deep semantic analysis. As the only widely used logographic script, Chinese haThe numerous synonyms and near-synonyms, making hate slang diverse and hard to detect. Chinese hate expressions often evade detection through homophones, character splitting/combining , or historical allusions To tackle this, we collected common hate slang from real online forums and built the first annotated Chinese hate slang lexicon.

The ComMA Dataset v0.2

The ComMA Dataset v0.2 is a multilingual dataset annotated with a hierarchical, fine-grained tagset marking different types of aggression and the "context" in which they occur. The context, here, is defined by the conversational thread in which a specific comment occurs and also the "type" of discursive role that the comment is performing with respect to the previous comment. The initial dataset, being discussed here (and made available as part of the ComMA@ICON shared task), consists of a total 15,000 annotated comments in four languages - Meitei, Bangla, Hindi, and Indian English - collected from various social media platforms such as YouTube, Facebook, Twitter and Telegram. As is usual on social media websites, a large number of these comments are multilingual, mostly code-mixed with English.

ViTHSD

A Vietnamese dataset for hate speech detection by the specific target. The dataset contains 10,000 comments, each comment has 05 targets with three relevant hateful levels.

TUN-EL

Arabic multi-dialectal hate speech dataset. It consists of 23,033 tweets labelled for Hate, Abusive and Normal in three Arabic dialects Egyptian, Lebaneese and Tunisian. How to cite this work: Badri, Nabil, Ferihane Kboubi, and Anja Habacha Chaibi. "Towards automatic detection of inappropriate content in multi-dialectic Arabic text." International Conference on Computational Collective Intelligence. Cham: Springer International Publishing, 2022.

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.