Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning algorithms for mean field optimal control

H. Mete Soner, Josef Teichmann, Qinxin Yan

TL;DR

This work develops a neural-network based numerical method for mean-field optimal control by approximating the mean-field problem with a single large N-particle system that exploits exchangeability. The control policy can depend on time, an agent's state, and the empirical distribution of the population, with universal approximation on Wasserstein spaces enabling flexible policy representations. The authors prove convergence to the mean-field limit using propagation of chaos and bound the randomness via Rademacher complexity, providing a rigorous foundation for the method. Numerical experiments on Linear Quadratic, Kuramoto, and systemic risk models demonstrate accuracy, phase-transition behavior, and robustness across parameters and initial distributions, highlighting the approach's computational efficiency and applicability to high-dimensional, distribution-dependent control problems.

Abstract

We analyze an algorithm to numerically solve the mean-field optimal control problems by approximating the optimal feedback controls using neural networks with problem specific architectures. We approximate the model by an $N$-particle system and leverage the exchangeability of the particles to obtain substantial computational efficiency. In addition to several numerical examples, a convergence analysis is provided. We also developed a universal approximation theorem on Wasserstein spaces.

Learning algorithms for mean field optimal control

TL;DR

This work develops a neural-network based numerical method for mean-field optimal control by approximating the mean-field problem with a single large N-particle system that exploits exchangeability. The control policy can depend on time, an agent's state, and the empirical distribution of the population, with universal approximation on Wasserstein spaces enabling flexible policy representations. The authors prove convergence to the mean-field limit using propagation of chaos and bound the randomness via Rademacher complexity, providing a rigorous foundation for the method. Numerical experiments on Linear Quadratic, Kuramoto, and systemic risk models demonstrate accuracy, phase-transition behavior, and robustness across parameters and initial distributions, highlighting the approach's computational efficiency and applicability to high-dimensional, distribution-dependent control problems.

Abstract

We analyze an algorithm to numerically solve the mean-field optimal control problems by approximating the optimal feedback controls using neural networks with problem specific architectures. We approximate the model by an -particle system and leverage the exchangeability of the particles to obtain substantial computational efficiency. In addition to several numerical examples, a convergence analysis is provided. We also developed a universal approximation theorem on Wasserstein spaces.

Paper Structure

This paper contains 26 sections, 13 theorems, 97 equations, 6 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notations
  3. Control problems
  4. Mean field optimal control problem
  5. $N$-Particle System
  6. Numerical algorithm
  7. Neural controls
  8. Algorithm
  9. Universal Approximation on $\mathop{\mathrm{\mathcal{X}}}\nolimits$
  10. Numerical results
  11. Linear quadratic problem
  12. Kuramoto mean field control problem
  13. Sub-critical case
  14. Super-critical case
  15. Generalization of the neural network
  16. ...and 11 more sections

Key Result

Theorem 3.1

Let $\rho = 1 + \|.\|^2$ be the weight function of quadratic growth on $E \subset \mathbb{R}^d$ and $R(\mu)$ the second moment of a measure $\mu \in \mathcal{P}_2(E)$, a weight function on Wasserstein space equipped with the weak topology. Then $(x,\mu) \mapsto \rho \vee R := \max(\rho(x), R(\mu))$

Figures (6)

  • Figure 1: Linear Quadratic MFC
  • Figure 2: Initial distribution: uniform distribution
  • Figure 3: Initial distribution: Two-cluster distribution
  • Figure 4: Initial distribution: uniform distribution
  • Figure 5: Initial distribution: Two-cluster distribution
  • ...and 1 more figures

Theorems & Definitions (31)

  • Remark 2.1
  • Theorem 3.1
  • Remark 3.1
  • Remark 3.2
  • Example 1
  • Theorem 4.1
  • Remark 4.1
  • Example 2
  • Theorem 5.1: phase transition CCS
  • Remark 5.1
  • ...and 21 more