FairPair: A Robust Evaluation of Biases in Language Models through Paired Perturbations

TL;DR

FairPair introduces a grounded counterfactual evaluation framework for measuring differential treatment in language outputs by pairing prompts with the same demographic and accounting for sampling variability. It defines a bias metric $\mathcal{F}(x)=\frac{\mathbb{B}^2(x)}{\mathbb{V}_{gp}(x)\mathbb{V}_{pg}(x)}$ based on ground-truth continuations $g(x)$ and $g(p(x))$, repeated sampling, and two scoring functions (sentiment and token dissimilarity). Evaluations on Common Sents across six configurations reveal statistically significant differential treatment, with larger models showing different bias patterns and qualitative analyses highlighting a shift from occupation-centric language to family/hobby and personality traits when perturbing gender. The approach is extensible to other demographics and scoring functions, offering robustness beyond extreme prompts, though it entails higher computational cost and careful perturbation design.

Abstract

The accurate evaluation of differential treatment in language models to specific groups is critical to ensuring a positive and safe user experience. An ideal evaluation should have the properties of being robust, extendable to new groups or attributes, and being able to capture biases that appear in typical usage (rather than just extreme, rare cases). Relatedly, bias evaluation should surface not only egregious biases but also ones that are subtle and commonplace, such as a likelihood for talking about appearances with regard to women. We present FairPair, an evaluation framework for assessing differential treatment that occurs during ordinary usage. FairPair operates through counterfactual pairs, but crucially, the paired continuations are grounded in the same demographic group, which ensures equivalent comparison. Additionally, unlike prior work, our method factors in the inherent variability that comes from the generation process itself by measuring the sampling variability. We present an evaluation of several commonly used generative models and a qualitative analysis that indicates a preference for discussing family and hobbies with regard to women.

Figures (6)

• Figure 1: An example construction of FairPairs, where the perturbation function being used is John (male) $\rightarrow$ Jane (female). Evaluation is conducted on the $p(g(x))$ (the perturbed original generation) and $g(p(x))$ (the perturbation generation), which are both grounded in the same entity (Jane).
• Figure 2: An illustration of the samples involved in calculating the bias $\mathbb B$, calculated between samples from $p(g(x))$ and $g(p(x))$ (solid arrows), and the sampling variability $\mathbb V$, calculated between samples within $p(g(x))$or$g(p(x))$ (dashed arrows). Prior work focuses primarily on the bias term without grounding in the same entity and without accounting for sampling variability; FairPair, on the other hand, addresses both these concerns.
• Figure 3: Bias according to Jaccard dissimilarity versus the number of samples (up to 500) of fairpairs used. For most models, values start to converge after about 300 samples.
• Figure 4: Bias according to Jaccard dissimilarity versus the number of folds $k$ used for 500 samples. For most models, values start to converge after $k=100$ (with each fold having 5 samples).
• Figure 5: Sampling variability ($\mathbb{V}_{pg}$ and $\mathbb{V}_{gp}$) and bias ($\mathbb{B}(x)$) for all baseline models using Jaccard dissimilarity. Larger models tend to have larger differences between sampling variability and bias, particularly for LLaMa and InstructGPT.