DefVerify: Do Hate Speech Models Reflect Their Dataset's Definition?

TL;DR

DefVerify tackles the problem of whether hate speech models faithfully reflect the dataset definitions used to train them. It introduces a three-step framework to encode definitions via Hate Speech Criteria, build an enriched diagnostic set from HateCheck, and assess alignment while diagnosing where misalignment originates; the approach is demonstrated across six widely-used English datasets. Although results reveal substantial gaps between dataset definitions and model behavior, DefVerify provides a practical protocol for diagnosing biases, annotation issues, and generalization problems prior to deployment. The work highlights the importance of explicit, public, and well-validated definitions and diagnostics to improve safety-critical NLP systems and offers a foundation for extending the methodology to multilingual and legally informed contexts.

Abstract

When building a predictive model, it is often difficult to ensure that application-specific requirements are encoded by the model that will eventually be deployed. Consider researchers working on hate speech detection. They will have an idea of what is considered hate speech, but building a model that reflects their view accurately requires preserving those ideals throughout the workflow of data set construction and model training. Complications such as sampling bias, annotation bias, and model misspecification almost always arise, possibly resulting in a gap between the application specification and the model's actual behavior upon deployment. To address this issue for hate speech detection, we propose DefVerify: a 3-step procedure that (i) encodes a user-specified definition of hate speech, (ii) quantifies to what extent the model reflects the intended definition, and (iii) tries to identify the point of failure in the workflow. We use DefVerify to find gaps between definition and model behavior when applied to six popular hate speech benchmark datasets.

Figures (9)

• Figure 1: Potential failure points when creating a hate speech dataset. All biases can accumulate in the model.
• Figure 2: DefVerify: Our proposed methodology to verify if model behavior is aligned with the dataset's intended definition. The figure depicts each individual step in the context of hate speech detection in our paper.
• Figure 3: Overview of which target groups are mentioned in the definition for each dataset.
• Figure 4: Accuracy for the different aspects on HateCheck, with the number of samples for each aspect in brackets. We separate the accuracy for aspects with both hate (h) and non-hate (nh) samples. SO: sexual orientation, GC: group characteristics, St: stereotype, Vi: incitement of violence, Di: incitement of discrimination.
• Figure 5: Accuracy for the different aspects on HateCheck, when removing the offensive classes from the Davidson, Founta, and HX datasets with fBERT.