GPT-HateCheck: Can LLMs Write Better Functional Tests for Hate Speech Detection?

TL;DR

This work addresses biases in hate speech detection benchmarks by introducing GPT-HateCheck, a generate-and-filter framework that uses large language models to create diverse, realistic functional tests aligned with HateCheck functionalities. The approach couples LLM-based test-case generation with NLI-based validation to ensure faithfulness to both the target functionality and gold labels, complemented by crowd and expert annotations. Empirical results show GPT-HateCheck produces more natural and varied test cases and reveals weaknesses in near-state-of-the-art detectors (e.g., HateBERT) that HateCheck misses, highlighting the value of fine-grained functional diagnostics. The framework enables targeted dataset augmentation and offers a path toward more robust hate speech detection systems, while acknowledging limitations such as reliance on closed-source LLMs and English-only data.

Abstract

Online hate detection suffers from biases incurred in data sampling, annotation, and model pre-training. Therefore, measuring the averaged performance over all examples in held-out test data is inadequate. Instead, we must identify specific model weaknesses and be informed when it is more likely to fail. A recent proposal in this direction is HateCheck, a suite for testing fine-grained model functionalities on synthesized data generated using templates of the kind "You are just a [slur] to me." However, despite enabling more detailed diagnostic insights, the HateCheck test cases are often generic and have simplistic sentence structures that do not match the real-world data. To address this limitation, we propose GPT-HateCheck, a framework to generate more diverse and realistic functional tests from scratch by instructing large language models (LLMs). We employ an additional natural language inference (NLI) model to verify the generations. Crowd-sourced annotation demonstrates that the generated test cases are of high quality. Using the new functional tests, we can uncover model weaknesses that would be overlooked using the original HateCheck dataset.

Figures (7)

• Figure 1: The overview of GPT-HateCheck. We first instantiate the prompt template with the target group in consideration and instruct GPT to generate candidate test cases. The test cases are then validated by an entailment model to ensure the generations conform with the functionality. In the example, although both generated messages are hateful towards the target group, the second one does not contain profanity and will be discarded.
• Figure 2: The prompt factory method that instantiates prompt templates w/ or w/o demonstrations.
• Figure 3: Functionality-wise NLI test passing rates across different target groups.
• Figure 4: Functionality-wise accuracy of HateBERT on the HateCheck and GPT-HateCheck test datasets. A lower accuracy indicates more challenging test cases.
• Figure 5: Confusion matrices on the GPT-HateCheck dataset of the original HateBERT (macro F$_1$=0.70) and HateBERT fine-tuned using ToxiGen dataset (macro F$_1$=0.33).