Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM-augmented empirical game theoretic simulation for social-ecological systems

Jennifer Shi, Christopher K. Frantz, Christian Kimmich, Saba Siddiki, Atrisha Sarkar

TL;DR

The paper investigates integrating large language models (LLMs) with empirical game-theoretic analysis (EGTA) to simulate governance in social-ecological systems (SES). It compares four frameworks—procedural ABMs, generative ABMs, naive LLM-EGTA, and expert-guided LLM-EGTA—using a real-world Amu Darya case to assess sustainability, wealth distribution, and robustness under centralized vs. decentralized governance. Key contributions include two LLM-augmented EGTA pipelines (naive and expert-guided), evidence that framework choice critically shapes dynamics (e.g., potential collapse under naive LLM-EGTA vs. sustained mixed economies with expert guidance and Pigouvian taxes), and a robustness analysis showing that expert-in-the-loop guidance and internalized externalities are essential for plausible SES outcomes. The work highlights the practical importance of combining formal equilibrium analysis with human expertise when embedding LLMs into SES simulations for policy analysis and institutional design.

Abstract

Designing institutions for social-ecological systems requires models that capture heterogeneity, uncertainty, and strategic interaction. Multiple modeling approaches have emerged to meet this challenge, including empirical game-theoretic analysis (EGTA), which merges ABM's scale and diversity with game-theoretic models' formal equilibrium analysis. The newly popular class of LLM-driven simulations provides yet another approach, and it is not clear how these approaches can be integrated with one another, nor whether the resulting simulations produce a plausible range of behaviours for real-world social-ecological governance. To address this gap, we compare four LLM-augmented frameworks: procedural ABMs, generative ABMs, LLM-EGTA, and expert guided LLM-EGTA, and evaluate them on a real-world case study of irrigation and fishing in the Amu Darya basin under centralized and decentralized governance. Our results show: first, procedural ABMs, generative ABMs, and LLM-augmented EGTA models produce strikingly different patterns of collective behaviour, highlighting the value of methodological diversity. Second, inducing behaviour through system prompts in LLMs is less effective than shaping behaviour through parameterized payoffs in an expert-guided EGTA-based model.

LLM-augmented empirical game theoretic simulation for social-ecological systems

TL;DR

The paper investigates integrating large language models (LLMs) with empirical game-theoretic analysis (EGTA) to simulate governance in social-ecological systems (SES). It compares four frameworks—procedural ABMs, generative ABMs, naive LLM-EGTA, and expert-guided LLM-EGTA—using a real-world Amu Darya case to assess sustainability, wealth distribution, and robustness under centralized vs. decentralized governance. Key contributions include two LLM-augmented EGTA pipelines (naive and expert-guided), evidence that framework choice critically shapes dynamics (e.g., potential collapse under naive LLM-EGTA vs. sustained mixed economies with expert guidance and Pigouvian taxes), and a robustness analysis showing that expert-in-the-loop guidance and internalized externalities are essential for plausible SES outcomes. The work highlights the practical importance of combining formal equilibrium analysis with human expertise when embedding LLMs into SES simulations for policy analysis and institutional design.

Abstract

Designing institutions for social-ecological systems requires models that capture heterogeneity, uncertainty, and strategic interaction. Multiple modeling approaches have emerged to meet this challenge, including empirical game-theoretic analysis (EGTA), which merges ABM's scale and diversity with game-theoretic models' formal equilibrium analysis. The newly popular class of LLM-driven simulations provides yet another approach, and it is not clear how these approaches can be integrated with one another, nor whether the resulting simulations produce a plausible range of behaviours for real-world social-ecological governance. To address this gap, we compare four LLM-augmented frameworks: procedural ABMs, generative ABMs, LLM-EGTA, and expert guided LLM-EGTA, and evaluate them on a real-world case study of irrigation and fishing in the Amu Darya basin under centralized and decentralized governance. Our results show: first, procedural ABMs, generative ABMs, and LLM-augmented EGTA models produce strikingly different patterns of collective behaviour, highlighting the value of methodological diversity. Second, inducing behaviour through system prompts in LLMs is less effective than shaping behaviour through parameterized payoffs in an expert-guided EGTA-based model.
Paper Structure (19 sections, 3 equations, 5 figures, 1 table)

This paper contains 19 sections, 3 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Simulation methodologies
  4. LLM-augmented EGTA
  5. Naive LLM-EGTA
  6. Expert-guided LLM-EGTA
  7. ABM-based approaches
  8. Case Study
  9. Simulation of ecological variables
  10. Behaviour models
  11. Decentralized models
  12. Procedural ABM:
  13. Generative ABM:
  14. Naive LLM-EGTA simulation:
  15. Exp LLM-EGTA simulation:
  16. ...and 4 more sections

Figures (5)

  • Figure 1: Internal structure of an action situation reproduced from Ostrom 2005 ostrom2005understanding
  • Figure 2: Pipeline for four different simulation methodologies. Panels (c) and (d) show two approaches to LLM augmentation within the EGTA pipeline: a naive approach that relies solely on LLMs at each step, and an expert-guided approach in which a designer refines the game model. The ecological ABM, which simulates the environmental and ecological variables, is integrated into all four approaches.
  • Figure 3: Map of Amu Darya Basin reproduced from schluter2007mechanisms.
  • Figure 4: Total budget (including farming and fishing) of each farming household from upstream (1) to downstream (9) over a period of 100 years under different simulation models and frameworks. $x$ axis: time in years, $y$ axis: household wealth.
  • Figure 5: Distribution of activities across farming households in the population over a period of 100 years under different simulation frameworks. Each horizontal bar is a stacked-bar chart showing the percentage of households ($y$ axis) that engage in each activity.