Simulating Opinion Dynamics with Networks of LLM-based Agents

TL;DR

This work investigates how to simulate human opinion dynamics using populations of large language model (LLM) agents, contrasting their capabilities with traditional agent-based models. It reveals an intrinsic bias in LLMs toward accurate, ground-truth information, which drives consensus aligned with scientific reality, and shows that prompting for confirmation bias can induce opinion fragmentation similar to ABM results. The study implements a dyadic interaction framework where agents maintain persona-driven memories and update beliefs through natural-language exchanges, across 15 topics and two memory schemes, evaluating framing and bias effects with Bias and Diversity metrics. While promising for capturing nuanced linguistic interactions, the findings also highlight limitations in reproducing diverse, fact-resistant viewpoints and underline the need for further tuning of LLMs with real-world discourse and more realistic network structures.

Abstract

Accurately simulating human opinion dynamics is crucial for understanding a variety of societal phenomena, including polarization and the spread of misinformation. However, the agent-based models (ABMs) commonly used for such simulations often over-simplify human behavior. We propose a new approach to simulating opinion dynamics based on populations of Large Language Models (LLMs). Our findings reveal a strong inherent bias in LLM agents towards producing accurate information, leading simulated agents to consensus in line with scientific reality. This bias limits their utility for understanding resistance to consensus views on issues like climate change. After inducing confirmation bias through prompt engineering, however, we observed opinion fragmentation in line with existing agent-based modeling and opinion dynamics research. These insights highlight the promise and limitations of LLM agents in this domain and suggest a path forward: refining LLMs with real-world discourse to better simulate the evolution of human beliefs.

Figures (6)

• Figure 1: Contrast between LLM generative agents and classic Agent-Based Models (ABMs). While both can simulate opinion dynamics, LLM generative agents use natural language for input ($x_{input}$) and output ($x_{output}$), maintain beliefs ($m^t$), and employ transformer-based LLM for belief updating. In contrast, classic ABMs use numerical values for input and output, maintain beliefs ($o^t$), and use hand-crafted equations for belief updating.
• Figure 2: (a) Schematic of the LLM agent network designed to simulate opinion dynamics across various topics, including global warming as a potential conspiracy. The network consists of agents, each role-playing a unique persona, with initial beliefs spanning acceptance, rejection, and neutrality regarding claims with known scientific consensus. Through the iterative cycles of writing and sharing tweets within their network connections, these agents' opinions evolve due to social influence. (b) An agent's memory $m_{i}^{t}$, including (1) initial persona, (2) optional closed-world restriction, (3) optional cognitive bias, and (4) historical events up to time $t$. Memory can be either cumulative (left) or reflective (right).
• Figure 3: Experimental setup for simulating opinion dynamics in agent interactions. At each time step $t$, agent $a_i$ writes a tweet $x_{i}^{t}$, which is subsequently presented to agent $a_j$. The agent $a_j$ then reports their thought $r_{j}^{t}$, which is processed by a classifier to yield a numerical opinion $o_{j}^{t}$. Both agents update their respective memory modules, $m_{i}^{t}$ and $m_{j}^{t}$, after writing or reviewing a tweet, which informs their future behaviors.
• Figure 4: Opinion trajectories $\langle o_i \rangle$ of LLM agents and the final opinion distribution $F_{o}^{T}$ on the topic of Global Warming. Panels (a) and (b) display the impact of cognitive biases under (a) false and (b) true framing conditions, respectively. Each row represents a different level of confirmation bias: no confirmation bias (top row), weak confirmation bias (middle row), and strong confirmation bias (bottom row). Panels (c) and (d) serve as baselines, with (c) being role-playing but with no interaction, and (d) being no role-playing and no interaction, respectively. The color of each line plot corresponds to the agent's initial opinion $o_i^{t=0}$: dark blue (+2), light blue (+1), grey (0), light red (-1), and dark red (-2), corresponding to opinions ranging from strongly agree to strongly disagree. The LLM agents in this figure use cumulative memory.
• Figure 5: Varying initial opinion distribution $F_{o}^T$ for the global warming debate. (a) All agents start with a strongly positive opinion. (b) $8$ agents start with a strongly positive opinion while $2$ with a strongly negative opinion (c) $8$ agents start with a strongly negative opinion while $2$ with a strongly positive opinion. (d) All agents start with a strongly negative opinion. The color of each line plot corresponds to the agent's initial opinion $o_i^{t=0}$: dark blue (+2), light blue (+1), grey (0), light red (-1), and dark red (-2), corresponding to opinions ranging from strongly agree to strongly disagree. The LLM agents in this figure use cumulative memory.