Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

I Am Not Them: Fluid Identities and Persistent Out-group Bias in Large Language Models

Wenchao Dong, Assem Zhunis, Hyojin Chin, Jiyoung Han, Meeyoung Cha

TL;DR

This work systematically probes cross-language and cross-model biases in large language models by prompting them with social-identity personas and validated individualism/collectivism surveys, extending the analysis to political domains. It demonstrates robust out-group bias that often exceeds in-group bias across six languages and three models, and shows that such biases persist under robustness checks (temperature, relaxed prompts, and embedded prompt histories). The study also reveals how persona framing can counteract or reveal underlying biases, and discusses mitigation approaches including prompt engineering and uncertainty control, while highlighting ethical and practical implications for cultural and political discourse. Overall, the findings underscore the need for continuous, cross-disciplinary evaluation of LLM bias and careful design of prompts to mitigate undesirable in-group/out-group dynamics in real-world deployments.

Abstract

We explored cultural biases-individualism vs. collectivism-in ChatGPT across three Western languages (i.e., English, German, and French) and three Eastern languages (i.e., Chinese, Japanese, and Korean). When ChatGPT adopted an individualistic persona in Western languages, its collectivism scores (i.e., out-group values) exhibited a more negative trend, surpassing their positive orientation towards individualism (i.e., in-group values). Conversely, when a collectivistic persona was assigned to ChatGPT in Eastern languages, a similar pattern emerged with more negative responses toward individualism (i.e., out-group values) as compared to collectivism (i.e., in-group values). The results indicate that when imbued with a particular social identity, ChatGPT discerns in-group and out-group, embracing in-group values while eschewing out-group values. Notably, the negativity towards the out-group, from which prejudices and discrimination arise, exceeded the positivity towards the in-group. The experiment was replicated in the political domain, and the results remained consistent. Furthermore, this replication unveiled an intrinsic Democratic bias in Large Language Models (LLMs), aligning with earlier findings and providing integral insights into mitigating such bias through prompt engineering. Extensive robustness checks were performed using varying hyperparameter and persona setup methods, with or without social identity labels, across other popular language models.

I Am Not Them: Fluid Identities and Persistent Out-group Bias in Large Language Models

TL;DR

This work systematically probes cross-language and cross-model biases in large language models by prompting them with social-identity personas and validated individualism/collectivism surveys, extending the analysis to political domains. It demonstrates robust out-group bias that often exceeds in-group bias across six languages and three models, and shows that such biases persist under robustness checks (temperature, relaxed prompts, and embedded prompt histories). The study also reveals how persona framing can counteract or reveal underlying biases, and discusses mitigation approaches including prompt engineering and uncertainty control, while highlighting ethical and practical implications for cultural and political discourse. Overall, the findings underscore the need for continuous, cross-disciplinary evaluation of LLM bias and careful design of prompts to mitigate undesirable in-group/out-group dynamics in real-world deployments.

Abstract

We explored cultural biases-individualism vs. collectivism-in ChatGPT across three Western languages (i.e., English, German, and French) and three Eastern languages (i.e., Chinese, Japanese, and Korean). When ChatGPT adopted an individualistic persona in Western languages, its collectivism scores (i.e., out-group values) exhibited a more negative trend, surpassing their positive orientation towards individualism (i.e., in-group values). Conversely, when a collectivistic persona was assigned to ChatGPT in Eastern languages, a similar pattern emerged with more negative responses toward individualism (i.e., out-group values) as compared to collectivism (i.e., in-group values). The results indicate that when imbued with a particular social identity, ChatGPT discerns in-group and out-group, embracing in-group values while eschewing out-group values. Notably, the negativity towards the out-group, from which prejudices and discrimination arise, exceeded the positivity towards the in-group. The experiment was replicated in the political domain, and the results remained consistent. Furthermore, this replication unveiled an intrinsic Democratic bias in Large Language Models (LLMs), aligning with earlier findings and providing integral insights into mitigating such bias through prompt engineering. Extensive robustness checks were performed using varying hyperparameter and persona setup methods, with or without social identity labels, across other popular language models.
Paper Structure (26 sections, 12 figures, 12 tables)

This paper contains 26 sections, 12 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Probing Human-like Bias in LLMs
  4. Group Membership and Inter-group Dynamics
  5. Cultural Orientations and Political Polarization
  6. Out-group Bias in Cultural Domain
  7. Methodology
  8. Experimental Settings
  9. Results
  10. Out-group Cultural Bias
  11. Robustness Analysis
  12. Temperature
  13. Persona Relaxation
  14. Survey Replication
  15. Persona Format Sensitivity
  16. ...and 11 more sections

Figures (12)

  • Figure 1: Experimental design and bias computation method. For each language, we assess the leaning with and without setting a persona. Language-consistent cultural value questions represent the in-group bias (e.g., agreement to individualism statements for Western languages), and language-inconsistent cultural value questions represent the out-group bias (e.g., agreement to collectivism statements for Western languages).
  • Figure 2: ChatGPT's cultural bias is displayed in six panels, one for each language. Black circles indicate the averaged response scores obtained from a 9-point Likert scale in response to collectivism questions (represented by circles connected by red arrows) or individualism questions (represented by circles connected by blue arrows). The direction of the arrows points to the results before and after setting the personas. Individualism-enforcing personas were set to Western languages, and collectivism-enforcing personas were set to Eastern languages.
  • Figure 3: Political in-group and out-group biases for ChatGPT assessment in English. Black points indicate the agreement level with and without setting either a Democratic (left) or Republican (right) persona, as assessed on a 6-point Likert scale. Blue arrows show in-group bias, and red arrows show out-group bias.
  • Figure 4: Cultural in-group and out-group biases for Gemini (a) and Llama (b) across six languages with temperature set to 1. Black circles indicate the averaged response scores obtained from a 9-point Likert scale in response to collectivism questions (represented by circles connected by red arrows) or individualism questions (represented by circles connected by blue arrows). The direction of the arrows points to the results before and after setting the personas. Individualism-enforcing personas were set to Western languages, and collectivism-enforcing personas were set to Eastern languages.
  • Figure 5: Cultural in-group and out-group biases for ChatGPT across six languages with temperature set to 0. Black circles indicate the averaged response scores obtained from a 9-point Likert scale in response to collectivism questions (represented by circles connected by red arrows) or individualism questions (represented by circles connected by blue arrows). The direction of the arrows points to the results before and after setting the personas. Individualism-enforcing personas were set to Western languages, and collectivism-enforcing personas were set to Eastern languages.
  • ...and 7 more figures