Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mean-Field Games with Constraints

Anran Hu, Zijiu Lyu

TL;DR

The work extends mean-field game theory to Constrained Mean-Field Games (CMFGs) where each agent solves a constrained Markov decision process over a finite horizon. It develops an existence theory under a strict feasibility assumption and a uniqueness result under Lasry–Lions monotonicity, and introduces Constrained Mean Field Occupation Measure Optimization (CMFOMO) to compute equilibria via a single convex optimization with occupation measures and KKT conditions. The paper also analyzes population-level constraints, shows the connection to agent-population-level constraints via twinned costs, and proves that CMFG equilibria yield $O(1/\sqrt{N})$-Nash equilibria for finite networks. Numerical experiments on a constrained SIS model demonstrate the framework’s effectiveness across constraint types and highlight the trade-offs between feasibility and optimality in practice.

Abstract

This paper introduces a framework of Constrained Mean-Field Games (CMFGs), where each agent solves a constrained Markov decision process (CMDP). This formulation captures scenarios in which agents' strategies are subject to feasibility, safety, or regulatory restrictions, thereby extending the scope of classical mean field game (MFG) models. We first establish the existence of CMFG equilibria under a strict feasibility assumption, and we further show uniqueness under a classical monotonicity condition. To compute equilibria, we develop Constrained Mean-Field Occupation Measure Optimization (CMFOMO), an optimization-based scheme that parameterizes occupation measures and shows that finding CMFG equilibria is equivalent to solving a single optimization problem with convex constraints and bounded variables. CMFOMO does not rely on uniqueness of the equilibria and can approximate all equilibria with arbitrary accuracy. We further prove that CMFG equilibria induce $O(1 / \sqrt{N})$-Nash equilibria in the associated constrained $N$-player games, thereby extending the classical justification of MFGs as approximations for large but finite systems. Numerical experiments on a modified Susceptible-Infected-Susceptible (SIS) epidemic model with various constraints illustrate the effectiveness and flexibility of the framework.

Mean-Field Games with Constraints

TL;DR

The work extends mean-field game theory to Constrained Mean-Field Games (CMFGs) where each agent solves a constrained Markov decision process over a finite horizon. It develops an existence theory under a strict feasibility assumption and a uniqueness result under Lasry–Lions monotonicity, and introduces Constrained Mean Field Occupation Measure Optimization (CMFOMO) to compute equilibria via a single convex optimization with occupation measures and KKT conditions. The paper also analyzes population-level constraints, shows the connection to agent-population-level constraints via twinned costs, and proves that CMFG equilibria yield -Nash equilibria for finite networks. Numerical experiments on a constrained SIS model demonstrate the framework’s effectiveness across constraint types and highlight the trade-offs between feasibility and optimality in practice.

Abstract

This paper introduces a framework of Constrained Mean-Field Games (CMFGs), where each agent solves a constrained Markov decision process (CMDP). This formulation captures scenarios in which agents' strategies are subject to feasibility, safety, or regulatory restrictions, thereby extending the scope of classical mean field game (MFG) models. We first establish the existence of CMFG equilibria under a strict feasibility assumption, and we further show uniqueness under a classical monotonicity condition. To compute equilibria, we develop Constrained Mean-Field Occupation Measure Optimization (CMFOMO), an optimization-based scheme that parameterizes occupation measures and shows that finding CMFG equilibria is equivalent to solving a single optimization problem with convex constraints and bounded variables. CMFOMO does not rely on uniqueness of the equilibria and can approximate all equilibria with arbitrary accuracy. We further prove that CMFG equilibria induce -Nash equilibria in the associated constrained -player games, thereby extending the classical justification of MFGs as approximations for large but finite systems. Numerical experiments on a modified Susceptible-Infected-Susceptible (SIS) epidemic model with various constraints illustrate the effectiveness and flexibility of the framework.

Paper Structure

This paper contains 19 sections, 23 theorems, 100 equations, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Our work and approach.
  3. Related work.
  4. Organization of the paper.
  5. Constrained Mean-Field Games
  6. Population-level constraints and agent-population-level constraints.
  7. Mean-Field Games with Agent-Population-Level Constraints
  8. Existence and Uniqueness of Nash Equilibrium in CMFGs
  9. Optimization Framework for Solving NE in CMFGs
  10. Population-Level Constrained Mean-Field Games
  11. Optimization Framework for Solving NE in CMFGs
  12. Connection between Population-Level and Agent-Population-Level Constraints
  13. Approximation of NE in Finite-player Games with Agent-population-level Constraints
  14. Numerical Experiments
  15. Agent-population-level constraint on state
  16. ...and 4 more sections

Key Result

Theorem 3.1

Under Assumptions ass: strict feasibility and ass: continuous, there exists at least one CMFG NE defined by Definition def: constrained ne.

Figures (5)

  • Figure 1: Optimized policy of the constrained SIS model when $\gamma_0 = 0.25$. Left: the evolution of $G_{\text{opt}}$ and $G_{\text{fea}}$ with respect to optimization iteration, where $G_{\text{opt}}^0$ represents the optimality gap of the initial policy; Right: the percentage of infected agents at each time step under the optimized policy, where the red dotted line represents the threshold $\gamma_0$ of the agent-population-level constraint \ref{['eq: gamma_0.25_con']}.
  • Figure 2: Sensitivity analysis under the agent-population-level constraint \ref{['eq: gamma_0.25_con']}. The $x$-axis represents the time steps and the $y$-axis represents the percentage of infected agents (Left) or agents who go out (Right). The policies are optimized when $\gamma_0$ takes different values.
  • Figure 3: Sensitivity analysis under the agent-population-level constraint \ref{['eq: agent_action']}. The $x$-axis represents the time steps and the $y$-axis represents the percentage of infected agents (Left) or agents who go out (Right). The policies are optimized when $\gamma_0$ takes different values.
  • Figure 4: Evolution of the objective function \ref{['eq: pop cmfomo obj']} of CMFOMO defined in Theorem \ref{['thm: pop cmfomo']} under the population-level constraint \ref{['eq: macro_exp_con']}. The $x$-axis represents the optimization iteration, and the $y$-axis represents the objective value. Different lines correspond to different values of $\gamma_0$.
  • Figure 5: Mean-field flow induced by the optimized policy under the population-level constraint \ref{['eq: macro_exp_con']} with $\gamma_0 = 0.5$. The $x$-axis represents the time steps and the $y$-axis represents the percentage of infected agents (Left) or agents who go out (Right).

Theorems & Definitions (51)

  • Definition 1: Nash equilibrium of CMFGs
  • Remark 2.1
  • Example 1: Constrained SIS model
  • Remark 2.2: Lagrangian method and penalty method
  • Theorem 3.1: Existence of NE in CMFGs
  • Remark 3.1
  • Lemma 3.2: CMDPs as LPs
  • Proposition 3.3
  • Lemma 3.4: A bounded solution of $\mathcal{K}(L)$
  • proof
  • ...and 41 more