A Dataset for the Detection of Dehumanizing Language

TL;DR

The paper tackles the lack of dehumanization-focused data by releasing two English datasets drawn from OpenSubtitles and Common Crawl: a large unlabeled corpus and a smaller manually annotated set. It proposes a keyword-driven extraction and 5-sentence windowing approach, and evaluates a baseline valence/keyword detector against a fine-tuned HateBERT model. Inter-annotator agreement is reported, and analysis shows that HateBERT outperforms the baseline, though performance is uneven across patterns and data sources. By providing these datasets, the work supports future automated dehumanization detection and analysis in political and media discourse.

Abstract

Dehumanization is a mental process that enables the exclusion and ill treatment of a group of people. In this paper, we present two data sets of dehumanizing text, a large, automatically collected corpus and a smaller, manually annotated data set. Both data sets include a combination of political discourse and dialogue from movie subtitles. Our methods give us a broad and varied amount of dehumanization data to work with, enabling further exploratory analysis and automatic classification of dehumanization patterns. Both data sets will be publicly released.

Figures (1)

• Figure 1: Percentage of tokens extracted from each forum in Common Crawl