Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPT

Shucheng Zhu, Weikang Wang, Ying Liu

TL;DR

This paper investigates nationality bias in ChatGPT (GPT-3.5) by generating 4,680 open-ended nationality discourses across 195 countries in Chinese and English, under multiple prompts and temperatures. It introduces a continuum-based bias evaluation framework combining automated metrics, human pairwise annotations, and self-assessment by the model, enabling nuanced detection beyond binary biased/unbiased labels. The study finds that outputs are predominantly positive and non-offensive but still reflect real-world nationality biases, with cross-language differences indicating cultural divergence in bias. Compared to GPT-2, ChatGPT shows reduced hate speech and more favorable sentiment, though negative prompts can elicit biased content, underscoring the need for ongoing scrutiny and cross-lingual fairness considerations in LLMs.

Abstract

While nationality is a pivotal demographic element that enhances the performance of language models, it has received far less scrutiny regarding inherent biases. This study investigates nationality bias in ChatGPT (GPT-3.5), a large language model (LLM) designed for text generation. The research covers 195 countries, 4 temperature settings, and 3 distinct prompt types, generating 4,680 discourses about nationality descriptions in Chinese and English. Automated metrics were used to analyze the nationality bias, and expert annotators alongside ChatGPT itself evaluated the perceived bias. The results show that ChatGPT's generated discourses are predominantly positive, especially compared to its predecessor, GPT-2. However, when prompted with negative inclinations, it occasionally produces negative content. Despite ChatGPT considering its generated text as neutral, it shows consistent self-awareness about nationality bias when subjected to the same pair-wise comparison annotation framework used by human annotators. In conclusion, while ChatGPT's generated texts seem friendly and positive, they reflect the inherent nationality biases in the real world. This bias may vary across different language versions of ChatGPT, indicating diverse cultural perspectives. The study highlights the subtle and pervasive nature of biases within LLMs, emphasizing the need for further scrutiny.

Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPT

TL;DR

This paper investigates nationality bias in ChatGPT (GPT-3.5) by generating 4,680 open-ended nationality discourses across 195 countries in Chinese and English, under multiple prompts and temperatures. It introduces a continuum-based bias evaluation framework combining automated metrics, human pairwise annotations, and self-assessment by the model, enabling nuanced detection beyond binary biased/unbiased labels. The study finds that outputs are predominantly positive and non-offensive but still reflect real-world nationality biases, with cross-language differences indicating cultural divergence in bias. Compared to GPT-2, ChatGPT shows reduced hate speech and more favorable sentiment, though negative prompts can elicit biased content, underscoring the need for ongoing scrutiny and cross-lingual fairness considerations in LLMs.

Abstract

While nationality is a pivotal demographic element that enhances the performance of language models, it has received far less scrutiny regarding inherent biases. This study investigates nationality bias in ChatGPT (GPT-3.5), a large language model (LLM) designed for text generation. The research covers 195 countries, 4 temperature settings, and 3 distinct prompt types, generating 4,680 discourses about nationality descriptions in Chinese and English. Automated metrics were used to analyze the nationality bias, and expert annotators alongside ChatGPT itself evaluated the perceived bias. The results show that ChatGPT's generated discourses are predominantly positive, especially compared to its predecessor, GPT-2. However, when prompted with negative inclinations, it occasionally produces negative content. Despite ChatGPT considering its generated text as neutral, it shows consistent self-awareness about nationality bias when subjected to the same pair-wise comparison annotation framework used by human annotators. In conclusion, while ChatGPT's generated texts seem friendly and positive, they reflect the inherent nationality biases in the real world. This bias may vary across different language versions of ChatGPT, indicating diverse cultural perspectives. The study highlights the subtle and pervasive nature of biases within LLMs, emphasizing the need for further scrutiny.
Paper Structure (31 sections, 2 equations, 6 figures, 4 tables)

This paper contains 31 sections, 2 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Social Bias in NLP
  4. Nationality Bias
  5. Bias in Generation LLMs
  6. Bias Benchmark
  7. Methodology
  8. Text Generation
  9. Countries
  10. Prompts
  11. Discourse Generation
  12. Bias Evaluation
  13. Automated Metrics
  14. Vocabulary Richness (RC)
  15. Sentiment (SM)
  16. ...and 16 more sections

Figures (6)

  • Figure 1: RC, SM, OF and HS values of the texts generated by different prompts in four temperature settings in both Chinese (zh) and English (en).
  • Figure 2: The maximal (max-) and minimal (min-) RC values in the texts generated by GPT-2.
  • Figure 3: The maximal (max-) and minimal (min-) SM values in the texts generated by GPT-2.
  • Figure 4: The maximal (max-) and minimal (min-) HS values in the texts generated by GPT-2.
  • Figure 5: The evaluation results by human annotators (-hum) and ChatGPT (-ChatGPT) in Chinese and English with different temperature settings. The closer the color of a country is to deep red, the more biased the text generated by this country is. The closer the color is to deep green, the less biased.
  • ...and 1 more figures