Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

People will agree what I think: Investigating LLM's False Consensus Effect

Junhyuk Choi, Yeseon Hong, Bugeun Kim

TL;DR

This paper investigates whether Large Language Models exhibit the False Consensus Effect (FCE) and how prompting styles influence its manifestation. Study 1 employs a psychology-informed, confounder-controlled design with cross-cultural and gender dimensions across four LLMs to establish that LLMs do show FCE, with cultural differences evident in several models and limited gender effects. Study 2 systematically manipulates prompts along provided-information and reasoning-chain dimensions, finding that opposite reasoning and deeper reasoning can reduce FCE, while certain information prompts can amplify it, though model-specific effects and ceiling phenomena occur. The work highlights that FCE in LLMs can be interpreted through established psychological methods, demonstrates generalizability beyond medical or domain-specific contexts, and offers practical guidance on prompt design to mitigate false consensus in interactive AI systems.

Abstract

Large Language Models (LLMs) have been recently adopted in interactive systems requiring communication. As the false belief in a model can harm the usability of such systems, LLMs should not have cognitive biases that humans have. Psychologists especially focus on the False Consensus Effect (FCE), a cognitive bias where individuals overestimate the extent to which others share their beliefs or behaviors, because FCE can distract smooth communication by posing false beliefs. However, previous studies have less examined FCE in LLMs thoroughly, which needs more consideration of confounding biases, general situations, and prompt changes. Therefore, in this paper, we conduct two studies to examine the FCE phenomenon in LLMs. In Study 1, we investigate whether LLMs have FCE. In Study 2, we explore how various prompting styles affect the demonstration of FCE. As a result of these studies, we identified that popular LLMs have FCE. Also, the result specifies the conditions when FCE becomes more or less prevalent compared to normal usage.

People will agree what I think: Investigating LLM's False Consensus Effect

TL;DR

This paper investigates whether Large Language Models exhibit the False Consensus Effect (FCE) and how prompting styles influence its manifestation. Study 1 employs a psychology-informed, confounder-controlled design with cross-cultural and gender dimensions across four LLMs to establish that LLMs do show FCE, with cultural differences evident in several models and limited gender effects. Study 2 systematically manipulates prompts along provided-information and reasoning-chain dimensions, finding that opposite reasoning and deeper reasoning can reduce FCE, while certain information prompts can amplify it, though model-specific effects and ceiling phenomena occur. The work highlights that FCE in LLMs can be interpreted through established psychological methods, demonstrates generalizability beyond medical or domain-specific contexts, and offers practical guidance on prompt design to mitigate false consensus in interactive AI systems.

Abstract

Large Language Models (LLMs) have been recently adopted in interactive systems requiring communication. As the false belief in a model can harm the usability of such systems, LLMs should not have cognitive biases that humans have. Psychologists especially focus on the False Consensus Effect (FCE), a cognitive bias where individuals overestimate the extent to which others share their beliefs or behaviors, because FCE can distract smooth communication by posing false beliefs. However, previous studies have less examined FCE in LLMs thoroughly, which needs more consideration of confounding biases, general situations, and prompt changes. Therefore, in this paper, we conduct two studies to examine the FCE phenomenon in LLMs. In Study 1, we investigate whether LLMs have FCE. In Study 2, we explore how various prompting styles affect the demonstration of FCE. As a result of these studies, we identified that popular LLMs have FCE. Also, the result specifies the conditions when FCE becomes more or less prevalent compared to normal usage.
Paper Structure (95 sections, 3 figures, 22 tables)

This paper contains 95 sections, 3 figures, 22 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Examining bias in LLMs
  4. Mitigating biases of LMs
  5. Study 1: Examining FCE of LLM
  6. Procedure
  7. Participants:
  8. Procedure:
  9. Tested LLMs
  10. Analysis
  11. Result and Discussion
  12. H1-1 (FCE):
  13. H1-2 (Culture)
  14. H1-3 (Gender)
  15. Study 2: Mitigating FCE with Prompt
  16. ...and 80 more sections

Figures (3)

  • Figure 1: Story 1, used in ross1977false. The highlighted bold-faced text shows two options in this story.
  • Figure 2: Four stories and queries used in ross1977false
  • Figure 3: Heatmap of FCE strength, representing interaction effect between two prompting styles, for each model.