Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Behavioral Alignment in LLM Social Simulations via Context Formation and Navigation

Letian Kong, Qianran, Jin, Renyu Zhang

TL;DR

This work identifies systematic misalignment of LLMs in complex, multi-agent decision tasks and proposes a two-stage framework: context formation (explicit task representation) and context navigation (guided reasoning within that representation). Through focal replication of Kremer & Debo, and extensions to Gui 2025 demand estimation and Casón 2024 signaling, the authors show that complex environments require both stages for human-aligned behavior across multiple SOTA LLMs, while simpler tasks may rely on context formation alone. The framework is validated across diverse decision environments and model families, demonstrating cross-task generalizability and offering a principled form of context engineering to complement human-subject research. The findings highlight when each stage is necessary and provide a practical blueprint for diagnosing and designing LLM social simulations that can complement or scale human behavioral studies.

Abstract

Large language models (LLMs) are increasingly used to simulate human behavior in experimental settings, but they systematically diverge from human decisions in complex decision-making environments, where participants must anticipate others' actions and form beliefs based on observed behavior. We propose a two-stage framework for improving behavioral alignment. The first stage, context formation, explicitly specifies the experimental design to establish an accurate representation of the decision task and its context. The second stage, context navigation, guides the reasoning process within that representation to make decisions. We validate this framework through a focal replication of a sequential purchasing game with quality signaling (Kremer and Debo, 2016), extending to a crowdfunding game with costly signaling (Cason et al., 2025) and a demand-estimation task (Gui and Toubia, 2025) to test generalizability across decision environments. Across four state-of-the-art (SOTA) models (GPT-4o, GPT-5, Claude-4.0-Sonnet-Thinking, DeepSeek-R1), we find that complex decision-making environments require both stages to achieve behavioral alignment with human benchmarks, whereas the simpler demand-estimation task requires only context formation. Our findings clarify when each stage is necessary and provide a systematic approach for designing and diagnosing LLM social simulations as complements to human subjects in behavioral research.

Improving Behavioral Alignment in LLM Social Simulations via Context Formation and Navigation

TL;DR

This work identifies systematic misalignment of LLMs in complex, multi-agent decision tasks and proposes a two-stage framework: context formation (explicit task representation) and context navigation (guided reasoning within that representation). Through focal replication of Kremer & Debo, and extensions to Gui 2025 demand estimation and Casón 2024 signaling, the authors show that complex environments require both stages for human-aligned behavior across multiple SOTA LLMs, while simpler tasks may rely on context formation alone. The framework is validated across diverse decision environments and model families, demonstrating cross-task generalizability and offering a principled form of context engineering to complement human-subject research. The findings highlight when each stage is necessary and provide a practical blueprint for diagnosing and designing LLM social simulations that can complement or scale human behavioral studies.

Abstract

Large language models (LLMs) are increasingly used to simulate human behavior in experimental settings, but they systematically diverge from human decisions in complex decision-making environments, where participants must anticipate others' actions and form beliefs based on observed behavior. We propose a two-stage framework for improving behavioral alignment. The first stage, context formation, explicitly specifies the experimental design to establish an accurate representation of the decision task and its context. The second stage, context navigation, guides the reasoning process within that representation to make decisions. We validate this framework through a focal replication of a sequential purchasing game with quality signaling (Kremer and Debo, 2016), extending to a crowdfunding game with costly signaling (Cason et al., 2025) and a demand-estimation task (Gui and Toubia, 2025) to test generalizability across decision environments. Across four state-of-the-art (SOTA) models (GPT-4o, GPT-5, Claude-4.0-Sonnet-Thinking, DeepSeek-R1), we find that complex decision-making environments require both stages to achieve behavioral alignment with human benchmarks, whereas the simpler demand-estimation task requires only context formation. Our findings clarify when each stage is necessary and provide a systematic approach for designing and diagnosing LLM social simulations as complements to human subjects in behavioral research.
Paper Structure (24 sections, 7 equations, 2 figures, 3 tables)

This paper contains 24 sections, 7 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Theoretical Framework for LLM Decision-Making
  3. Empirical Validation of Our Framework
  4. Literature Review
  5. Generative AI and LLM in Business Applications
  6. LLM Social Simulations
  7. LLM Alignment Challenges and Solutions
  8. Theoretical Framework
  9. Human Problem-Solving
  10. Context Formation and Context Navigation
  11. Simulation and Empirical Analysis
  12. Overview of the Experimental Setting
  13. Simulation Design
  14. Prompt Design
  15. Simulation Results
  16. ...and 9 more sections

Figures (2)

  • Figure 1: Prompting Methods for Baseline, Context Formation, and Context Formation and Navigation
  • Figure 2: Demand curve elicited from LLMs (GPT-4o)