HatePrototypes: Interpretable and Transferable Representations for Implicit and Explicit Hate Speech Detection

TL;DR

HatePrototypes tackles the challenge that hate speech detectors trained on explicit targets often fail to generalize to implicit hate. By constructing class prototypes μ_c^(ℓ) as layer-wise centroids of hidden representations and using a parameter-free, margin-based exit strategy, the approach enables cross-task transfer between explicit and implicit hate benchmarks and accelerates inference without fine-tuning. Empirical results show prototypes built from as few as 50 examples per class can achieve strong cross-domain performance and that prototype-guided early exiting yields substantial speed-ups with minimal accuracy loss, across multiple model families and datasets. The work also demonstrates practical benefits for guardrail safety models and provides code and resources to support broader exploration of efficient, transferable hate speech detection in real-world moderation scenarios.

Abstract

Optimization of offensive content moderation models for different types of hateful messages is typically achieved through continued pre-training or fine-tuning on new hate speech benchmarks. However, existing benchmarks mainly address explicit hate toward protected groups and often overlook implicit or indirect hate, such as demeaning comparisons, calls for exclusion or violence, and subtle discriminatory language that still causes harm. While explicit hate can often be captured through surface features, implicit hate requires deeper, full-model semantic processing. In this work, we question the need for repeated fine-tuning and analyze the role of HatePrototypes, class-level vector representations derived from language models optimized for hate speech detection and safety moderation. We find that these prototypes, built from as few as 50 examples per class, enable cross-task transfer between explicit and implicit hate, with interchangeable prototypes across benchmarks. Moreover, we show that parameter-free early exiting with prototypes is effective for both hate types. We release the code, prototype resources, and evaluation scripts to support future research on efficient and transferable hate speech detection.

Figures (6)

• Figure 1: Accuracy on IHC implicit hate types and neutral class obtained with IHC prototypes from models fine-tuned on 4 considered benchmarks.
• Figure 2: F1-scores for BERT and OPT models across four dataset pairs (tuned source-evaluation target) with varying numbers of prototypes per class.
• Figure 3: Prototype selection results: relative cross-dataset transfer F1-scores for BERT and OPT (right). Values show how much of each model’s in-domain performance carries over to other datasets.
• Figure 4: Layer-wise proportion of samples exiting under the prototype-based early-exit criterion for the explicit HX and implicit SBIC benchmarks using the OPT model.
• Figure 5: F1-scores vs. speed-up on HateXplain and SBIC for the OPT model, compared to entropy-based (DEEOPT) and patience-based (PABEE) baselines.