Interacting Large Language Model Agents. Interpretable Models and Social Learning

TL;DR

The paper develops an interpretable, theory‑driven framework for interacting Large Language Model Agents (LLMAs) by blending Bayesian inference, microeconomic rationality, and stochastic control. It first models a single LLMA as a rationally inattentive Bayesian utility maximizer (RIBUM) and shows how to reconstruct interpretable utilities from black‑box LLMAs; then extends to sequences of LLMAs performing Bayesian social learning, revealing conditions under which information cascades and herding occur. To counteract undesirable herding, the authors formulate two stochastic control settings—central control and incentivized autonomous LLMAs—and prove threshold structures for optimal stopping policies, complemented by a policy‑gradient algorithm that operates without full model knowledge. Numerical experiments on hate speech detection and product quality identification with LLaMA and ChatGPT demonstrate the approach’s ability to yield interpretable state estimation and active control of information sharing. The work provides reproducible code and points to broad applications in finance, online moderation, and personalized recommendations, with future directions toward broader LLMA networks and human‑in‑the‑loop integration.

Abstract

This paper discusses the theory and algorithms for interacting large language model agents (LLMAs) using methods from statistical signal processing and microeconomics. While both fields are mature, their application to decision-making involving interacting LLMAs remains unexplored. Motivated by Bayesian sentiment analysis on online platforms, we construct interpretable models and algorithms that enable LLMAs to interact and perform Bayesian inference. Because interacting LLMAs learn from both prior decisions and external inputs, they can exhibit bias and herding behavior. Thus, developing interpretable models and stochastic control algorithms is essential to understand and mitigate these behaviors. This paper has three main results. First, we show using Bayesian revealed preferences from microeconomics that an individual LLMA satisfies the necessary and sufficient conditions for rationally inattentive (bounded rationality) Bayesian utility maximization and, given an observation, the LLMA chooses an action that maximizes a regularized utility. Second, we utilize Bayesian social learning to construct interpretable models for LLMAs that interact sequentially with each other and the environment while performing Bayesian inference. Our proposed models capture the herding behavior exhibited by interacting LLMAs. Third, we propose a stochastic control framework to delay herding and improve state estimation accuracy under 2 settings: (a) centrally controlled LLMAs (b) autonomous LLMAs with incentives. We demonstrate the effectiveness of our methods on real datasets for hate speech classification and product quality assessment, using open-source models like LLaMA and closed-source models like ChatGPT. The main takeaway of this paper, based on empirical analysis and mathematical formalism, is that LLMAs act as rationally bounded Bayesian agents that exhibit social learning when interacting.

Figures (20)

• Figure 1: Summary of the proposed contributions: We discuss the different blackbox models for LLMA and how LLMAs can be used as a sensing mechanism to perform Bayesian inference. Part 1 models the LLMAs as a rationally inattentive Bayesian utility maximizer and numerically establishes the behavior in applications of product quality identification and hate speech classification. Part 2 discusses how Bayesian social learning in a sequence of LLMAs can be used for sequential state estimation. However, in Part 3, we show that the agents can perform the same incorrect action due to herding. We then discuss a stochastic control approach to delay herding when LLMAs are centrally controlled and when they are autonomous but are incentivized.
• Figure 2: Engineering with large language model agents (LLMAs): We propose engineering with LLMs on three different levels: a) First, we construct LLMAs with an LLM attached to the Bayesian engine. The LLM acts as a sensor for the text input and outputs interpretable low-dimensional outputs, which are used by the Bayesian engine to produce a state estimate. b) Second, we formulate necessary and sufficient conditions for a LLMAs to be a rationally inattentive Bayesian utility maximizer (RIBUM). We also present algorithms to reconstruct feasible utilities and rational inattention costs if the LLMA is indeed a RIBUM, attributing the LLMA with an interpretable microeconomic model. c) Finally, we demonstrate how a sequence of LLMAs can efficiently perform sequential Bayesian social learning by controlling their outputs to delay herding optimally. Our Bayesian social learning models can be extended to study Bayesian social learning in a network of LLMAs.
• Figure 3: Organization of the paper: The paper is divided into three parts. Part 1 deals with interpretable models for an individual LLM agent. Part 2 extends the models to a social learning setting where LLM agents interact with each other to perform Bayesian inference. Part 3 proposes stochastic control methods to delay herding in a sequence of LLM agents.
• Figure 4: Brief schematic of a large language model agent as a sensing mechanism for Bayesian Inference: LLM Input is composed of the system instruction prompt, which is also the control; the user prompt, which is a private observation; and the in-context examples generated from the previous LLM agents are the past actions. Based on the input, the LLM outputs an intermediate textual output. The Bayesian engine uses a likelihood function and past actions to select an action maximizing the expected utility. If utility function is not explicitly given, Bayesian revealed preference is used to obtain a set-valued estimate using an input-output dataset. The paper discusses variations of this model with application in Bayesian sentiment analysis.
• Figure 5: LLMAs can be used to detect and analyze the change in financial indicators (difference of close prices) by parsing financial news to extract $16$ interpretable features in Example \ref{['ex:financial']}. We analyze the news articles from $03/2020$ to $08/2020$ corresponding to the AAPL stock. We query the LLM for $16$ binary with different features, including whether the news article indicates optimism about the market and whether there is investor interest in the stock. The interpretable features can be used to analyze the stock (and subsequently for Bayesian inference), as illustrated by the difference in the ratio of the stock prices across days.

Theorems & Definitions (42)

• Example 1
• Remark 1
• Remark 2
• Remark 3
• Remark 4
• Remark 5
• Example 2
• Remark 6
• Remark 7
• Remark 8