Token Sampling Uncertainty Does Not Explain Homogeneity Bias in Large Language Models
Messi H. J. Lee, Soyeon Jeon
TL;DR
The paper investigates whether homogeneity bias in large language models stems from token-level sampling uncertainty. By analyzing entropy and differentiation probability across six models and four demographic groups, the authors find that sampling uncertainty does not consistently explain the observed bias, even as cosines of sentence embeddings reveal robust homogeneity for marginalized groups. This implies that inference-time interventions like temperature scaling are unlikely to mitigate the bias, and instead mitigation should target representational learning and training data composition. The work provides a mechanistic audit across contemporary LLMs and emphasizes the importance of structural, data- and representation-level remedies for equitable language technologies.
Abstract
Homogeneity bias is one form of stereotyping in AI models where certain groups are represented as more similar to each other than other groups. This bias is a major obstacle to creating equitable language technologies. We test whether the bias is driven by systematic differences in token-sampling uncertainty across six large language models. While we observe the presence of homogeneity bias using sentence similarity, we find very little difference in token sampling uncertainty across groups. This finding elucidates why temperature-based sampling adjustments fail to mitigate homogeneity bias. It suggests researchers should prioritize interventions targeting representation learning mechanisms and training corpus composition rather than inference-time output manipulations.