Probability of Differentiation Reveals Brittleness of Homogeneity Bias in GPT-4

TL;DR

This work introduces an encoder-free method to assess homogeneity bias in LLMs by prompting GPT-4 to output single words or expressions across 18 situation cues and measuring variability with probability of differentiation. Across five studies, the authors demonstrate extreme heterogeneity ($I^2$ \ge 99.9\%$) and no consistent race- or gender-based differences, challenging prior encoder-centric findings. Ablation analyses reveal that bias is highly brittle, varying with model version, identity signaling, and prompt specificity, suggesting past results may reflect encoder or prompt artifacts rather than robust LLM biases. The study advocates for open-model investigations and examination of linguistic features and longer-text generation to understand how homogeneity bias manifests, with implications for fair representation in AI outputs.

Abstract

Homogeneity bias in Large Language Models (LLMs) refers to their tendency to homogenize the representations of some groups compared to others. Previous studies documenting this bias have predominantly used encoder models, which may have inadvertently introduced biases. To address this limitation, we prompted GPT-4 to generate single word/expression completions associated with 18 situation cues-specific, measurable elements of environments that influence how individuals perceive situations and compared the variability of these completions using probability of differentiation. This approach directly assessed homogeneity bias from the model's outputs, bypassing encoder models. Across five studies, we find that homogeneity bias is highly volatile across situation cues and writing prompts, suggesting that the bias observed in past work may reflect those within encoder models rather than LLMs. Furthermore, we find that homogeneity bias in LLMs is brittle, as even minor and arbitrary changes in prompts can significantly alter the expression of biases. Future work should further explore how variations in syntactic features and topic choices in longer text generations influence homogeneity bias in LLMs.

Figures (5)

• Figure 1: A visualization of the study design. A completion prompt is supplied to the LLM, and the completions are used to compute probability of differentiation. In this example, completions for "Victoria" (top) are more evenly distributed across sports categories, yielding a higher probability of differentiation (0.80), while those for "Tanisha" (bottom) are more concentrated, resulting in a lower value (0.55).
• Figure 2: Probability of Differentiation of the four racial/ ethnic groups across the 18 situation cues. The error bars indicate 95% confidence intervals.
• Figure 3: Probability of Differentiation of the two gender groups across the 18 situation cues. The error bars indicate 95% confidence intervals.
• Figure A1: Probability of Differentiation of the four racial/ ethnic groups across eight areas of human experience.
• Figure A2: Probability of Differentiation of the two gender groups across eight areas of human experience.