Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bridging Expectation Signals: LLM-Based Experiments and a Behavioral Kalman Filter Framework

Yu Wang, Xiangchen Liu

TL;DR

This work develops a Behavioral Kalman Filter (BKF) to quantify how LLM-based economic agents update future expectations under dual micro and macro signals. By extending the standard Kalman Filter with a prior discount factor $\alpha$ and a subjective signal correlation $\rho$, and estimating via Bayesian linear regression on a 720-trial experiment with Household and Firm CEO personas, the authors uncover systematic biases: weights on priors and signals deviate from unity, micro signals are weighted more by households while macro signals dominate for CEOs, and a negative interaction between concurrent signals reveals cognitive interference. Across architectures, these patterns persist, though model-specific differences exist (e.g., loss aversion in GPT-4o). LoRA fine-tuning reduces some biases, particularly the interaction effect for households, but does not fully restore rational, additive updating, highlighting intrinsic limitations in current LLM expectation formation for high-stakes economic simulations.

Abstract

As LLMs increasingly function as economic agents, the specific mechanisms LLMs use to update their belief with heterogeneous signals remain opaque. We design experiments and develop a Behavioral Kalman Filter framework to quantify how LLM-based agents update expectations, acting as households or firm CEOs, update expectations when presented with individual and aggregate signals. The results from experiments and model estimation reveal four consistent patterns: (1) agents' weighting of priors and signals deviates from unity; (2) both household and firm CEO agents place substantially larger weights on individual signals compared to aggregate signals; (3) we identify a significant and negative interaction between concurrent signals, implying that the presence of multiple information sources diminishes the marginal weight assigned to each individual signal; and (4) expectation formation patterns differ significantly between household and firm CEO agents. Finally, we demonstrate that LoRA fine-tuning mitigates, but does not fully eliminate, behavioral biases in LLM expectation formation.

Bridging Expectation Signals: LLM-Based Experiments and a Behavioral Kalman Filter Framework

TL;DR

This work develops a Behavioral Kalman Filter (BKF) to quantify how LLM-based economic agents update future expectations under dual micro and macro signals. By extending the standard Kalman Filter with a prior discount factor and a subjective signal correlation , and estimating via Bayesian linear regression on a 720-trial experiment with Household and Firm CEO personas, the authors uncover systematic biases: weights on priors and signals deviate from unity, micro signals are weighted more by households while macro signals dominate for CEOs, and a negative interaction between concurrent signals reveals cognitive interference. Across architectures, these patterns persist, though model-specific differences exist (e.g., loss aversion in GPT-4o). LoRA fine-tuning reduces some biases, particularly the interaction effect for households, but does not fully restore rational, additive updating, highlighting intrinsic limitations in current LLM expectation formation for high-stakes economic simulations.

Abstract

As LLMs increasingly function as economic agents, the specific mechanisms LLMs use to update their belief with heterogeneous signals remain opaque. We design experiments and develop a Behavioral Kalman Filter framework to quantify how LLM-based agents update expectations, acting as households or firm CEOs, update expectations when presented with individual and aggregate signals. The results from experiments and model estimation reveal four consistent patterns: (1) agents' weighting of priors and signals deviates from unity; (2) both household and firm CEO agents place substantially larger weights on individual signals compared to aggregate signals; (3) we identify a significant and negative interaction between concurrent signals, implying that the presence of multiple information sources diminishes the marginal weight assigned to each individual signal; and (4) expectation formation patterns differ significantly between household and firm CEO agents. Finally, we demonstrate that LoRA fine-tuning mitigates, but does not fully eliminate, behavioral biases in LLM expectation formation.
Paper Structure (15 sections, 8 equations, 2 figures, 5 tables)

This paper contains 15 sections, 8 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Theoretical Framework: Behavioral Kalman Filter
  3. Standard Kalman Filter Baseline
  4. Behavioral Kalman Filter: Model Definition
  5. Empirical Strategy: Bayesian Reduced-Form Regression
  6. LLM-Based Experimental Design
  7. Scenario Matrix and Signal Construction
  8. Persona Implementation and Economic Motivation
  9. Template Structure and Output Standardization
  10. Model Selection and Computational Setup
  11. Experiments Results
  12. Descriptive Statistics and Main Trends
  13. Parameter Identification: Bayesian Linear Regression
  14. Can LoRA Fine-Tuning Eliminate Behavioral Biases in LLMs?
  15. Conclusion

Figures (2)

  • Figure 1: The identity priming text for both agent types. These contexts are prepended to the numerical shocks to test for persona-induced cognitive biases in information integration.
  • Figure 2: The modular experimental interface. By enforcing a strict JSON output with a Rationale field, we capture both the quantitative belief update and the qualitative weighting of signals.