Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humans

TL;DR

This study identifies a new bias in large language models: subordinate social groups are depicted as more homogeneous than dominant groups. Using ChatGPT, it analyzes eight intersectional group prompts and quantifies homogeneity with cosine similarities of sentence embeddings across multiple embedding models, finding robust main effects for race/ethnicity and gender, plus a significant interaction. While topical alignment explains part of the bias, it does not fully account for it, suggesting additional factors such as semantic and syntactic patterns learned from training data. The work highlights potential risks of stereotype reinforcement and urges development of mitigation strategies that address intersectional representation biases in LLMs. Overall, it provides a methodological framework to quantify perceived group homogeneity in generated text and emphasizes the importance of fairness and inclusivity in AI text generation.

Abstract

Large language models (LLMs) are becoming pervasive in everyday life, yet their propensity to reproduce biases inherited from training data remains a pressing concern. Prior investigations into bias in LLMs have focused on the association of social groups with stereotypical attributes. However, this is only one form of human bias such systems may reproduce. We investigate a new form of bias in LLMs that resembles a social psychological phenomenon where socially subordinate groups are perceived as more homogeneous than socially dominant groups. We had ChatGPT, a state-of-the-art LLM, generate texts about intersectional group identities and compared those texts on measures of homogeneity. We consistently found that ChatGPT portrayed African, Asian, and Hispanic Americans as more homogeneous than White Americans, indicating that the model described racial minority groups with a narrower range of human experience. ChatGPT also portrayed women as more homogeneous than men, but these differences were small. Finally, we found that the effect of gender differed across racial/ethnic groups such that the effect of gender was consistent within African and Hispanic Americans but not within Asian and White Americans. We argue that the tendency of LLMs to describe groups as less diverse risks perpetuating stereotypes and discriminatory behavior.

Figures (9)

• Figure 1: Mean standardized cosine similarity values of the four racial/ ethnic groups using BERT$_{-2}$. Error bars were omitted as confidence intervals were all smaller than 0.001.
• Figure 2: Standardized cosine similarity values of the two gender groups using BERT$_{-2}$. Error bars were omitted as confidence intervals were all smaller than 0.001.
• Figure 3: Standardized cosine similarity values of all eight intersectional groups using BERT$_{-2}$. Error bars were omitted as confidence intervals were all smaller than 0.001.
• Figure 4: Standardized cosine similarity values of all eight intersectional groups using all seven model specifications. Error bars were omitted as confidence intervals were all smaller than 0.001.
• Figure A1: Top five highest probability words of the 15 topics identified within the ChatGPT-generated text. Note that the textProcessor performs stemming which causes words like "adversity" and "adverse" to all show up as "advers".