Robust mean-field control under common noise uncertainty

TL;DR

This work develops a discrete-time, robust mean-field control framework under common noise uncertainty, where a social planner optimizes open-loop controls on an infinite horizon with discount $\beta$ to maximize a worst-case expected reward over an uncertainty set $\mathcal{Q}$. It advances the theory by lifting the problem to the space of probability measures, formulating a robust MDP with lifted state $\mu_t$ and lifted action $\Lambda_t$, and introducing a Bellman–Isaacs operator $\mathcal{T}$ that admits a unique fixed point $\overline{V}^*$. A propagation-of-chaos result shows that finite-$N$ robust cooperative dynamics converge to the robust MFC problem as $N\to\infty$, with explicit Wasserstein-rate bounds $M_N$ ensuring that $|V^N-V|=O(M_N)$. The paper also proves a verification theorem linking the lifted and original problems, and demonstrates equivalence between open-loop and closed-loop formulations, including the existence of optimal stationary closed-loop policies. Numerical experiments in distribution matching and systemic financial risk highlight the practical value of accounting for common-noise uncertainty, showing performance gains for moderate perturbations and illustrating the trade-offs of extreme robustness.

Abstract

We propose and analyze a framework for discrete-time robust mean-field control problems under common noise uncertainty. In this framework, the mean-field interaction describes the collective behavior of infinitely many cooperative agents' state and action, while the common noise -- a random disturbance affecting all agents' state dynamics -- is uncertain. A social planner optimizes over open-loop controls on an infinite horizon to maximize the representative agent's worst-case expected reward, where worst-case corresponds to the most adverse probability measure among all candidates inducing the unknown true law of the common noise process. We refer to this optimization as a robust mean-field control problem under common noise uncertainty. We first show that this problem arises as the asymptotic limit of a cooperative $N$-agent robust optimization problem, commonly known as propagation of chaos. We then prove the existence of an optimal open-loop control by linking the robust mean field control problem to a lifted robust Markov decision problem on the space of probability measures and by establishing the dynamic programming principle and Bellman--Isaac fixed point theorem for the lifted robust Markov decision problem. Finally, we complement our theoretical results with numerical experiments motivated by distribution planning and systemic risk in finance, highlighting the advantages of accounting for common noise uncertainty.

Figures (6)

• Figure 1: Values achieved under $p_{\textrm{true}}$ when using the optimal policy for the MFC under $p_{\textrm{ref}}$ (red dashed line) or the robust MFC under the uncertainty level $\delta_{\textrm{perturb}} \in \{0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2\}$ (blue curve) in Example 1. Shaded areas represents $\pm$ standard deviation over 8 independent runs.
• Figure 2: Three sample trajectories of the population distribution and corresponding action distribution for each state in Example 1. The target distribution to be matched is shown by dashed red lines.
• Figure 3: The three trajectories of common noise associated with Figure \ref{['fig:distrib-matching-traj']}
• Figure 4: Value achieved under $p_{\textrm{true}}$ when using the optimal policy for the MFC with $p_{\textrm{ref}}$ (red dashed line) or the optimal policy for the robust MFC with $\delta_{\textrm{perturb}} \in \{0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6\}$ (blue curve) in Example 2. Shaded areas represents $\pm$ standard deviation over 8 independent runs.
• Figure 5: Three sample trajectories of the population distribution and corresponding action distribution for each state in Example 2.

Theorems & Definitions (36)

• Definition 2.1: Filtrations
• Definition 2.2: Measures
• Remark 2.3
• Definition 2.4
• Definition 2.5: $N$-agent model
• Lemma 2.8
• Theorem 2.9
• Remark 2.10
• Definition 2.11
• Proposition 2.12