Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Mean-Field Games with Risk Aversion and Bounded Rationality

Bhavini Jeloka, Yue Guan, Panagiotis Tsiotras

Abstract

Recent advances in mean-field game literature enable the reduction of large-scale multi-agent problems to tractable interactions between a representative agent and a population distribution. However, existing approaches typically assume a fixed initial population distribution and fully rational agents, limiting robustness under distributional uncertainty and cognitive constraints. We address these limitations by introducing risk aversion with respect to the initial population distribution and by incorporating bounded rationality to model deviations from fully rational decision-making agents. The combination of these two elements yields a new and more general equilibrium concept, which we term the mean-field risk-averse quantal response equilibrium (MF-RQE). We establish existence results and prove convergence of fixed-point iteration and fictitious play to MF-RQE. Building on these insights, we develop a scalable reinforcement learning algorithm for scenarios with large state-action spaces. Numerical experiments demonstrate that MF-RQE policies achieve improved robustness relative to classical mean-field approaches that optimize expected cumulative rewards under a fixed initial distribution and are restricted to entropy-based regularizers.

Robust Mean-Field Games with Risk Aversion and Bounded Rationality

Abstract

Recent advances in mean-field game literature enable the reduction of large-scale multi-agent problems to tractable interactions between a representative agent and a population distribution. However, existing approaches typically assume a fixed initial population distribution and fully rational agents, limiting robustness under distributional uncertainty and cognitive constraints. We address these limitations by introducing risk aversion with respect to the initial population distribution and by incorporating bounded rationality to model deviations from fully rational decision-making agents. The combination of these two elements yields a new and more general equilibrium concept, which we term the mean-field risk-averse quantal response equilibrium (MF-RQE). We establish existence results and prove convergence of fixed-point iteration and fictitious play to MF-RQE. Building on these insights, we develop a scalable reinforcement learning algorithm for scenarios with large state-action spaces. Numerical experiments demonstrate that MF-RQE policies achieve improved robustness relative to classical mean-field approaches that optimize expected cumulative rewards under a fixed initial distribution and are restricted to entropy-based regularizers.
Paper Structure (20 sections, 13 theorems, 70 equations, 2 figures, 9 tables, 2 algorithms)

This paper contains 20 sections, 13 theorems, 70 equations, 2 figures, 9 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Notation
  4. Problem Formulation
  5. Mean-Field Games
  6. Risk due to Uncertainty in Initial Distributions
  7. Bounded Rationality in Mean-Field Games
  8. MF-RQE
  9. Learning MF-RQE
  10. Numerical Experiments
  11. Conclusions and Future Work
  12. Algorithm Pseudocodes
  13. Experiments
  14. Computation of pi*avg
  15. Experimental Details
  16. ...and 5 more sections

Key Result

Theorem 1

Let $\mathcal{X}$ be the set of functions mapping from a finite set $\Omega$ to $\mathbb{R}$. Then a mapping $\rho:\mathcal{X}\rightarrow \mathbb{R}$ is a convex risk measure if and only if there exists a penalty function $D:\mathcal{P}(\Omega)\rightarrow (-\infty,\infty]$ such that: $\rho(X)=\sup_{

Figures (2)

  • Figure 1: Performance of RQ-FPI and RQ-Fictitious Play across different environments and regularizers.
  • Figure 2: Convergence results of D-RQ-FPI with entropy regularization in the SIS and the Congestion scenarios.

Theorems & Definitions (33)

  • Definition 1
  • Definition 2: Identical Policy
  • Definition 3: Mean-Field
  • Definition 4: MFE huang2006large
  • Definition 5: Convex Risk Measures
  • Theorem 1: Dual Representation Theorem Risk_overview
  • Remark 1
  • Definition 6: MF-RQE
  • Proposition 1
  • Definition 7
  • ...and 23 more