Mind the (Belief) Gap: Group Identity in the World of LLMs

TL;DR

The paper investigates belief congruence as a social-psychology trait and its amplification in LLM-based multi-agent systems. It adapts Rokeach’s race-belief paradigm to simulate inter-group dynamics among LLM agents and evaluates effects on misinformation dissemination and learning. Empirical results show LLMs exhibit stronger belief congruence than humans, leading to increased misinformation spread and hindered learning, but three psychology-inspired mitigations—contact, accuracy nudges, and global citizenship—significantly reduce negative effects. By integrating social psychology with AI, the work provides practical guidance and opens-source resources to manage belief-driven biases in real-world AI applications.

Abstract

Social biases and belief-driven behaviors can significantly impact Large Language Models (LLMs) decisions on several tasks. As LLMs are increasingly used in multi-agent systems for societal simulations, their ability to model fundamental group psychological characteristics remains critical yet under-explored. In this study, we present a multi-agent framework that simulates belief congruence, a classical group psychology theory that plays a crucial role in shaping societal interactions and preferences. Our findings reveal that LLMs exhibit amplified belief congruence compared to humans, across diverse contexts. We further investigate the implications of this behavior on two downstream tasks: (1) misinformation dissemination and (2) LLM learning, finding that belief congruence in LLMs increases misinformation dissemination and impedes learning. To mitigate these negative impacts, we propose strategies inspired by: (1) contact hypothesis, (2) accuracy nudges, and (3) global citizenship framework. Our results show that the best strategies reduce misinformation dissemination by up to 37% and enhance learning by 11%. Bridging social psychology and AI, our work provides insights to navigate real-world interactions using LLMs while addressing belief-driven biases.

Figures (21)

• Figure 1: Campus study by rokeach1966race: A White participant with a specific belief interacts with four confederates:two White (one with a similar belief, one opposing) and two Black (one with a similar belief, one opposing). The participant selects two confederates to join them for coffee and explains their choice. We simulate this using a multi-agent LLM framework.
• Figure 2: Campus study (top) and Field study (bottom) simulations. s+o+ refer to choosing confederates belonging to s(imilar) and o(pposite) races with the same (+) views. There are six possible choice combinations (s+o+, s-o-, s+o-, s-o+, s+s-, and o+o-).
• Figure 3: Demographic and Belief Ablations: Results averaged across all settings of both campus and field studies. For demographic ablations, all models show high belief congruence. For belief ablation, agents show higher preference to those with similar political beliefs over those of the same race.
• Figure 4: Misinformation Dissemination: (heterogeneous group shown) involves political personas interacting and determining the veracity of a news item.
• Figure 5: Misinformation Dissemination Results: We show initial correctness rates (before interaction) and final correctness rates (after interaction) for both datasets in hom(ogeneous) dem(ocrat) and rep(ublican) settings. Correctness rates go down after interactions, showing increase in misinformation dissemination.