Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MetaHate: A Dataset for Unifying Efforts on Hate Speech Detection

Paloma Piot, Patricia Martín-Rodilla, Javier Parapar

TL;DR

This paper addresses the fragmentation of hate speech datasets by constructing MetaHate, a meta-collection aggregating 36 English, human-authored social-media datasets into a unified benchmark with about 1.2 million deduplicated posts. It details a rigorous data acquisition and curation process, along with descriptive and linguistic analyses (lexical patterns, NER, topic modeling, emotion analysis) and establishes baseline classifiers (SVM, CNN, BERT) to evaluate cross-dataset utility. The findings show that approximately 20% of posts are hateful, and that a BERT-based approach provides the strongest performance, underscoring MetaHate’s value for building robust hate-speech detectors across diverse sources. Overall, the work delivers a public resource and benchmarking framework that can accelerate research into context-aware, multilingual hate speech detection in online media.

Abstract

Hate speech represents a pervasive and detrimental form of online discourse, often manifested through an array of slurs, from hateful tweets to defamatory posts. As such speech proliferates, it connects people globally and poses significant social, psychological, and occasionally physical threats to targeted individuals and communities. Current computational linguistic approaches for tackling this phenomenon rely on labelled social media datasets for training. For unifying efforts, our study advances in the critical need for a comprehensive meta-collection, advocating for an extensive dataset to help counteract this problem effectively. We scrutinized over 60 datasets, selectively integrating those pertinent into MetaHate. This paper offers a detailed examination of existing collections, highlighting their strengths and limitations. Our findings contribute to a deeper understanding of the existing datasets, paving the way for training more robust and adaptable models. These enhanced models are essential for effectively combating the dynamic and complex nature of hate speech in the digital realm.

MetaHate: A Dataset for Unifying Efforts on Hate Speech Detection

TL;DR

This paper addresses the fragmentation of hate speech datasets by constructing MetaHate, a meta-collection aggregating 36 English, human-authored social-media datasets into a unified benchmark with about 1.2 million deduplicated posts. It details a rigorous data acquisition and curation process, along with descriptive and linguistic analyses (lexical patterns, NER, topic modeling, emotion analysis) and establishes baseline classifiers (SVM, CNN, BERT) to evaluate cross-dataset utility. The findings show that approximately 20% of posts are hateful, and that a BERT-based approach provides the strongest performance, underscoring MetaHate’s value for building robust hate-speech detectors across diverse sources. Overall, the work delivers a public resource and benchmarking framework that can accelerate research into context-aware, multilingual hate speech detection in online media.

Abstract

Hate speech represents a pervasive and detrimental form of online discourse, often manifested through an array of slurs, from hateful tweets to defamatory posts. As such speech proliferates, it connects people globally and poses significant social, psychological, and occasionally physical threats to targeted individuals and communities. Current computational linguistic approaches for tackling this phenomenon rely on labelled social media datasets for training. For unifying efforts, our study advances in the critical need for a comprehensive meta-collection, advocating for an extensive dataset to help counteract this problem effectively. We scrutinized over 60 datasets, selectively integrating those pertinent into MetaHate. This paper offers a detailed examination of existing collections, highlighting their strengths and limitations. Our findings contribute to a deeper understanding of the existing datasets, paving the way for training more robust and adaptable models. These enhanced models are essential for effectively combating the dynamic and complex nature of hate speech in the digital realm.
Paper Structure (8 sections, 6 figures, 2 tables)

This paper contains 8 sections, 6 figures, 2 tables.

Table of Contents

  1. Introduction and Motivation
  2. Data Acquisition and Preparation
  3. Meta-Collection Overview
  4. Dataset analysis
  5. Baselines
  6. Discussion and Conclusions
  7. Ethical Statement
  8. Appendix

Figures (6)

  • Figure 1: Comparison of Named Entity Type percentages between hate and non-hate posts.
  • Figure 2: Word clouds of hate topics (up) and non-hate topics (down).
  • Figure 3: t-SNE diagram illustrating the clustering of hate speech (left) and non-hate speech (right).
  • Figure 4: Radar plot showing the percentage of posts that contain a word associated with the Plutchick emotions for hate and non-hate data.
  • Figure 5: Distribution of pronouns in hate and non-hate posts (left). Hate speech tends to mention YOU more, possibly indicating attacks. On the other hand, non-hate speech more commonly includes references to I and HE, SHE, IT. And distribution of verb tenses in hate and non-hate speech posts (right). Note that PRESENT tenses are more prevalent in hate posts, suggesting possible attacks.
  • ...and 1 more figures