Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM-Guided Reinforcement Learning with Representative Agents for Traffic Modeling

Hanlin Sun, Jiayang Li

TL;DR

This work proposes to model each homogeneous traveler group facing the same decision context with a single representative LLM agent who behaves like the population's average, maintaining and updating a mixed strategy over routes that coincides with the group's aggregate flow proportions.

Abstract

Large language models (LLMs) are increasingly used as behavioral proxies for self-interested travelers in agent-based traffic models. Although more flexible and generalizable than conventional models, the practical use of these approaches remains limited by scalability due to the cost of calling one LLM for every traveler. Moreover, it has been found that LLM agents often make opaque choices and produce unstable day-to-day dynamics. To address these challenges, we propose to model each homogeneous traveler group facing the same decision context with a single representative LLM agent who behaves like the population's average, maintaining and updating a mixed strategy over routes that coincides with the group's aggregate flow proportions. Each day, the LLM reviews the travel experience and flags routes with positive reinforcement that they hope to use more often, and an interpretable update rule then converts this judgment into strategy adjustments using a tunable (progressively decaying) step size. The representative-agent design improves scalability, while the separation of reasoning from updating clarifies the decision logic while stabilizing learning. In classic traffic assignment settings, we find that the proposed approach converges rapidly to the user equilibrium. In richer settings with income heterogeneity, multi-criteria costs, and multi-modal choices, the generated dynamics remain stable and interpretable, reproducing plausible behavioral patterns well-documented in psychology and economics, for example, the decoy effect in toll versus non-toll road selection, and higher willingness-to-pay for convenience among higher-income travelers when choosing between driving, transit, and park-and-ride options.

LLM-Guided Reinforcement Learning with Representative Agents for Traffic Modeling

TL;DR

This work proposes to model each homogeneous traveler group facing the same decision context with a single representative LLM agent who behaves like the population's average, maintaining and updating a mixed strategy over routes that coincides with the group's aggregate flow proportions.

Abstract

Large language models (LLMs) are increasingly used as behavioral proxies for self-interested travelers in agent-based traffic models. Although more flexible and generalizable than conventional models, the practical use of these approaches remains limited by scalability due to the cost of calling one LLM for every traveler. Moreover, it has been found that LLM agents often make opaque choices and produce unstable day-to-day dynamics. To address these challenges, we propose to model each homogeneous traveler group facing the same decision context with a single representative LLM agent who behaves like the population's average, maintaining and updating a mixed strategy over routes that coincides with the group's aggregate flow proportions. Each day, the LLM reviews the travel experience and flags routes with positive reinforcement that they hope to use more often, and an interpretable update rule then converts this judgment into strategy adjustments using a tunable (progressively decaying) step size. The representative-agent design improves scalability, while the separation of reasoning from updating clarifies the decision logic while stabilizing learning. In classic traffic assignment settings, we find that the proposed approach converges rapidly to the user equilibrium. In richer settings with income heterogeneity, multi-criteria costs, and multi-modal choices, the generated dynamics remain stable and interpretable, reproducing plausible behavioral patterns well-documented in psychology and economics, for example, the decoy effect in toll versus non-toll road selection, and higher willingness-to-pay for convenience among higher-income travelers when choosing between driving, transit, and park-and-ride options.

Paper Structure

This paper contains 29 sections, 15 equations, 7 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Conventional traffic modeling
  3. LLM-driven day-to-day models
  4. Our contribution
  5. Organization
  6. Related Studies
  7. Day-to-day models in transportation
  8. Reinforcement learning models in behavioral economics
  9. LLM-driven strategic competition
  10. The Proposed Framework
  11. Idea.
  12. Initialization
  13. Learning mechanisms
  14. A fully LLM-driven baseline learning mechanism
  15. A fully LLM-driven RL mechanism
  16. ...and 14 more sections

Figures (7)

  • Figure 1: The 3N4L network.
  • Figure 2: The Hearn network.
  • Figure 3: Evolution of the relative equilibrium gap under different learning mechanisms across three benchmark networks.
  • Figure 4: A three-route tolling network.
  • Figure 5: Evolution of route flows under the LLM-guided RL dynamics for the three highway-tolling settings (A--C), comparing scenarios with and without Road 3. Shaded areas represent the spread of the six retained simulation runs (after filtering out four outliers), while solid lines denote their mean trajectories.
  • ...and 2 more figures

Theorems & Definitions (3)

  • Definition 3.1
  • Remark 3.2: Another interpretation of Rule 2
  • Remark 3.3: Requirement block