Learning dynamics from online-offline systems of LLM agents

TL;DR

This work investigates how different types of offline events, along with the"personalities" assigned to the LLMs, affect the network dynamics of online information spread of the events among the LLMs, and introduces a stochastic agent-based network model and a system of differential equations arising from a mean-field approximation to the agent-based model.

Abstract

Online information is increasingly linked to real-world instability, especially as automated accounts and LLM-based agents help spread and amplify news. In this work, we study how information spreads on networks of Large Language Models (LLMs) using mathematical models. We investigate how different types of offline events, along with the "personalities" assigned to the LLMs, affect the network dynamics of online information spread of the events among the LLMs. We introduce two models: 1) a stochastic agent-based network model and 2) a system of differential equations arising from a mean-field approximation to the agent-based model. We fit these models to simulations of the spread of armed-conflict news on social media, using LLM agents each with one of 32 personality trait profiles on k-regular random networks. Our results indicate that, despite the complexity of the news events, personalities, and LLM behaviors, the overall dynamics of the system are well described by a Susceptible-Infected (SI) type model with two transmission rates.

Figures (16)

• Figure 1: Overview of LLM simulation. LLM agents are connected through a network, each with a pre-assigned "personality", demonstrated with different colors in the figure. A seed ACLED news event is initially given to one LLM agent, which starts spreading the news. Then, in subsequent steps, the neighbors of the spreaders will randomly decide whether to share it with their neighboring LLMs in the network. The total engagement over time forms a sigmoidal curve (right panel), which we later model using either an agent-based stochastic process or a mean-field approximation.
• Figure 2: The distribution of events as clustered into $2$ groups by the first principal component of the text embeddings, suggested by possessing the highest Silhouette score. The clusters have distinct levels of severity: cluster $1$ contains more severe events, with the most common event-type labels being "political violence", "violence against civilians", and "attack". In contrast, events in cluster $2$ appear to be more peaceful, with the most common labels being "demonstrations", "protests", and "peaceful protest".
• Figure 3: Prompts to the LLMs that define personality behaviors and ask about their responses based on events. (a) The system-level prompt that defines the expected behaviors of the LLM's Big Five personality traits. (b) The agent-level prompt that gives the news text and asks the LLM to decide whether to share it with neighbors based on its personality. (c) The post-simulation prompt that asks the LLM to explain its decision.
• Figure 4: The left column presents ACLED event news descriptions for an example peaceful protest (top) and violent attack (bottom). News is shared with LLM agents via a prompt, and a decision is then made to share or not share the news with neighbors in the network (depending on the personality profile). On the right, examples of agents with different personalities are shown with their decisions and rationales.
• Figure 5: Demonstration of the process obtaining the response ratios for each personality profile for each given event to be used to later classify traits based on spreading behaviors. For a given event, agents with different personalities will each be prompted multiple times to retrieve a "yes" response ratio related to that event and that personality.