Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data-Driven Network LQG Mean Field Games with Heterogeneous Populations via Integral Reinforcement Learning

Jean Zhu, Shuang Gao

TL;DR

Under technical conditions on the persistency of excitation and on the existence of unique stabilizing solution to the corresponding AREs, the learned network-coupled MFG strategies are shown to converge to their true values.

Abstract

This paper establishes a data-driven solution for infinite horizon linear quadratic Gaussian Mean Field Games with network-coupled heterogeneous agent populations where the dynamics of the agents are unknown. The solution technique relies on Integral Reinforcement Learning and Kleinman's iteration for solving algebraic Riccati equations (ARE). The resulting algorithm uses trajectory data to generate network-coupled MFG strategies for agents and does not require parameters of agents' dynamics. Under technical conditions on the persistency of excitation and on the existence of unique stabilizing solution to the corresponding AREs, the learned network-coupled MFG strategies are shown to converge to their true values.

Data-Driven Network LQG Mean Field Games with Heterogeneous Populations via Integral Reinforcement Learning

TL;DR

Under technical conditions on the persistency of excitation and on the existence of unique stabilizing solution to the corresponding AREs, the learned network-coupled MFG strategies are shown to converge to their true values.

Abstract

This paper establishes a data-driven solution for infinite horizon linear quadratic Gaussian Mean Field Games with network-coupled heterogeneous agent populations where the dynamics of the agents are unknown. The solution technique relies on Integral Reinforcement Learning and Kleinman's iteration for solving algebraic Riccati equations (ARE). The resulting algorithm uses trajectory data to generate network-coupled MFG strategies for agents and does not require parameters of agents' dynamics. Under technical conditions on the persistency of excitation and on the existence of unique stabilizing solution to the corresponding AREs, the learned network-coupled MFG strategies are shown to converge to their true values.
Paper Structure (7 sections, 3 theorems, 56 equations, 2 figures, 3 tables, 1 algorithm)

This paper contains 7 sections, 3 theorems, 56 equations, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem formulation
  3. Data-driven multi-class LQG-MFG with completely unknown dynamics
  4. Numerical Example
  5. Conclusion
  6. Proof of proposition \ref{['prop:key_equations']}
  7. Lemma used in the Proof of Prop. \ref{['prop:key_equations']}

Key Result

Proposition 1

Under Assumptionsassum:A_B_stabi_A_Q_Obs and assum:Hamiltonian_Q, the MFG strategy of a generic agent $a_{k, i}$ in class $k\in\{1,..., \mathcal{K}\}$ exists and is uniquely given by where $P_k \succ 0$ and $s_k = \sum_{m=1}^{\mathcal{K}}\Pi_{km}\Bar{x}_m$ follow from the solutions to the following algebraic equations and the mean field dynamics with $\mathcal{P}\in \mathbb{R}^{N\times N}$, $\

Figures (2)

  • Figure 1: Convergence analysis of the IRL algorithm for local class-level results and global system-level results
  • Figure 2: Mean field trajectories under data-driven learned controls (left) and standard care() gains computed assuming known system and cost matrices (right)

Theorems & Definitions (7)

  • Remark 1
  • Remark 2
  • Proposition 1
  • Proposition 2
  • Remark 3
  • Remark 4: Trajectory Data Required by the Algorithm
  • Lemma 1