Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Primacy Effect of ChatGPT

Yiwei Wang, Yujun Cai, Muhao Chen, Yuxuan Liang, Bryan Hooi

TL;DR

The paper addresses whether ChatGPT inherits human primacy bias in discriminative NLU tasks by introducing label-order randomization in prompts and measuring prediction stability and label-index distributions. It leverages a label-shuffling methodology across diverse datasets (e.g., TACRED, TACRED-Revisit, Re-TACRED, Banking77, MASSIVE, GoEmotions, 20 Newsgroups) and compares ChatGPT with a BERT baseline, using a zero-temperature setting to minimize randomness. Key findings show a pronounced primacy effect in ChatGPT, with predictions shifting under label shuffling and a skew toward earlier labels, particularly on harder tasks; CoT does not fully mitigate this bias, raising fairness concerns quantified via Jensen–Shannon divergence. The study provides insights into reliability and fairness of ChatGPT-based NLU solutions and releases code to facilitate further research and mitigation strategies.

Abstract

Instruction-tuned large language models (LLMs), such as ChatGPT, have led to promising zero-shot performance in discriminative natural language understanding (NLU) tasks. This involves querying the LLM using a prompt containing the question, and the candidate labels to choose from. The question-answering capabilities of ChatGPT arise from its pre-training on large amounts of human-written text, as well as its subsequent fine-tuning on human preferences, which motivates us to ask: Does ChatGPT also inherits humans' cognitive biases? In this paper, we study the primacy effect of ChatGPT: the tendency of selecting the labels at earlier positions as the answer. We have two main findings: i) ChatGPT's decision is sensitive to the order of labels in the prompt; ii) ChatGPT has a clearly higher chance to select the labels at earlier positions as the answer. We hope that our experiments and analyses provide additional insights into building more reliable ChatGPT-based solutions. We release the source code at https://github.com/wangywUST/PrimacyEffectGPT.

Primacy Effect of ChatGPT

TL;DR

The paper addresses whether ChatGPT inherits human primacy bias in discriminative NLU tasks by introducing label-order randomization in prompts and measuring prediction stability and label-index distributions. It leverages a label-shuffling methodology across diverse datasets (e.g., TACRED, TACRED-Revisit, Re-TACRED, Banking77, MASSIVE, GoEmotions, 20 Newsgroups) and compares ChatGPT with a BERT baseline, using a zero-temperature setting to minimize randomness. Key findings show a pronounced primacy effect in ChatGPT, with predictions shifting under label shuffling and a skew toward earlier labels, particularly on harder tasks; CoT does not fully mitigate this bias, raising fairness concerns quantified via Jensen–Shannon divergence. The study provides insights into reliability and fairness of ChatGPT-based NLU solutions and releases code to facilitate further research and mitigation strategies.

Abstract

Instruction-tuned large language models (LLMs), such as ChatGPT, have led to promising zero-shot performance in discriminative natural language understanding (NLU) tasks. This involves querying the LLM using a prompt containing the question, and the candidate labels to choose from. The question-answering capabilities of ChatGPT arise from its pre-training on large amounts of human-written text, as well as its subsequent fine-tuning on human preferences, which motivates us to ask: Does ChatGPT also inherits humans' cognitive biases? In this paper, we study the primacy effect of ChatGPT: the tendency of selecting the labels at earlier positions as the answer. We have two main findings: i) ChatGPT's decision is sensitive to the order of labels in the prompt; ii) ChatGPT has a clearly higher chance to select the labels at earlier positions as the answer. We hope that our experiments and analyses provide additional insights into building more reliable ChatGPT-based solutions. We release the source code at https://github.com/wangywUST/PrimacyEffectGPT.
Paper Structure (13 sections, 1 equation, 4 figures, 2 tables)

This paper contains 13 sections, 1 equation, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Primacy Effect of ChatGPT
  3. Prompts for ChatGPT
  4. Analysis with Label Shuffling
  5. Prediction Comparison on an Instance
  6. Statistics of Predicted Indices
  7. Experiments
  8. Experiment Setup
  9. Consistency under Label Shuffling
  10. Primacy Effect of ChatGPT
  11. Evaluation on Fairness
  12. Related Work
  13. Conclusion

Figures (4)

  • Figure 1: Primacy Effect of ChatGPT: ChatGPT tends to return labels in earlier positions as the answer. This plot shows the distribution of ChatGPT's predicted label indices in TACRED (42 classes), where we randomly shuffle labels before every prediction (see Sec. \ref{['sec:analysis']}).
  • Figure 2: A prompt example for ChatGPT.
  • Figure 3: We analyze the primacy effects of ChatGPT by randomly shuffling the labels in the prompts.
  • Figure 4: The distribution of predicted indices of the test instances with label shuffling before every prediction.