Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Disentangling Hate Across Target Identities

Yiping Jin, Leo Wanner, Aneesh Moideen Koya

TL;DR

A study inspired by social psychology theory reveals that the accuracy of hatefulness prediction correlates strongly with the intensity of the stereotype, and quantitatively analyzes the impact of different factors on HS prediction.

Abstract

Hate speech (HS) classifiers do not perform equally well in detecting hateful expressions towards different target identities. They also demonstrate systematic biases in predicted hatefulness scores. Tapping on two recently proposed functionality test datasets for HS detection, we quantitatively analyze the impact of different factors on HS prediction. Experiments on popular industrial and academic models demonstrate that HS detectors assign a higher hatefulness score merely based on the mention of specific target identities. Besides, models often confuse hatefulness and the polarity of emotions. This result is worrisome as the effort to build HS detectors might harm the vulnerable identity groups we wish to protect: posts expressing anger or disapproval of hate expressions might be flagged as hateful themselves. We also carry out a study inspired by social psychology theory, which reveals that the accuracy of hatefulness prediction correlates strongly with the intensity of the stereotype.

Disentangling Hate Across Target Identities

TL;DR

A study inspired by social psychology theory reveals that the accuracy of hatefulness prediction correlates strongly with the intensity of the stereotype, and quantitatively analyzes the impact of different factors on HS prediction.

Abstract

Hate speech (HS) classifiers do not perform equally well in detecting hateful expressions towards different target identities. They also demonstrate systematic biases in predicted hatefulness scores. Tapping on two recently proposed functionality test datasets for HS detection, we quantitatively analyze the impact of different factors on HS prediction. Experiments on popular industrial and academic models demonstrate that HS detectors assign a higher hatefulness score merely based on the mention of specific target identities. Besides, models often confuse hatefulness and the polarity of emotions. This result is worrisome as the effort to build HS detectors might harm the vulnerable identity groups we wish to protect: posts expressing anger or disapproval of hate expressions might be flagged as hateful themselves. We also carry out a study inspired by social psychology theory, which reveals that the accuracy of hatefulness prediction correlates strongly with the intensity of the stereotype.

Paper Structure

This paper contains 23 sections, 1 equation, 12 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Hate Speech Detection Datasets
  4. Bias Analysis and Mitigation
  5. Methodology
  6. Datasets
  7. Models
  8. Disentangling Target Identity Mentions
  9. Disentangling Emotions
  10. Disentangling Stereotypes
  11. Stereotypes Interpretation
  12. Correlation with Hate Prediction
  13. Stereotype Phrase Analysis
  14. Conclusions and Future Work
  15. Examples from Datasets
  16. ...and 8 more sections

Figures (12)

  • Figure 1: Overview of our approach. We analyze target identity mentions' impact on hatefulness prediction in a minimal set experiment on the HateCheck dataset. We then extract fine-grained emotions and stereotypes from examples in GPT-HateCheck dataset to analyze the distributional difference among target identities and its impact on the classifiers' performance.
  • Figure 2: Normalized hatefulness predictions of models across target identities.
  • Figure 3: Frequent emotions detection in GPT-HateCheck dataset with at least ten occurrences.
  • Figure 4: Distribution of target identities for each detected emotion.
  • Figure 5: Kernel density estimate (KDE) in the warmth-competence semantic space of various target identities.
  • ...and 7 more figures