A Robust Bias Mitigation Procedure Based on the Stereotype Content Model

TL;DR

This work addresses the persistence of stereotype bias in contextualised language models by grounding a debiasing procedure in the Stereotype Content Model (SCM). It uses SCM-informed, two-axis (warmth/coldness and competence/incompetence) orthogonal debiasing to re-shape targeted identity embeddings via fine-tuning on BERT with a resource-efficient loss combining bias mitigation and regularisation terms. The approach reduces stereotyped associations across demographic groups, including intersectional ones, while largely preserving or even improving downstream performance on GLUE tasks, demonstrating practical applicability. The method is intended as a demographic-agnostic debiasing prototype that minimizes prior bias knowledge and computational overhead, enabling broad deployment across multilingual and multi-identity contexts.

Abstract

The Stereotype Content model (SCM) states that we tend to perceive minority groups as cold, incompetent or both. In this paper we adapt existing work to demonstrate that the Stereotype Content model holds for contextualised word embeddings, then use these results to evaluate a fine-tuning process designed to drive a language model away from stereotyped portrayals of minority groups. We find the SCM terms are better able to capture bias than demographic agnostic terms related to pleasantness. Further, we were able to reduce the presence of stereotypes in the model through a simple fine-tuning procedure that required minimal human and computer resources, without harming downstream performance. We present this work as a prototype of a debiasing procedure that aims to remove the need for a priori knowledge of the specifics of bias in the model.

Figures (1)

• Figure 1: Diagram of intended orthogonal projection of target terms away from the warmth dimension, determined by attribute terms in bold. EA names underlined