A fast iterative PDE-based algorithm for feedback controls of nonsmooth mean-field control problems

Abstract

We propose a PDE-based accelerated gradient algorithm for optimal feedback controls of McKean-Vlasov dynamics that involve mean-field interactions both in the state and action. The method exploits a forward-backward splitting approach and iteratively refines the approximate controls based on the gradients of smooth costs, the proximal maps of nonsmooth costs, and dynamically updated momentum parameters. At each step, the state dynamics is approximated via a particle system, and the required gradient is evaluated through a coupled system of nonlocal linear PDEs. The latter is solved by finite difference approximation or neural network-based residual approximation, depending on the state dimension. We present exhaustive numerical experiments for low and high-dimensional mean-field control problems, including sparse stabilization of stochastic Cucker-Smale models, which reveal that our algorithm captures important structures of the optimal feedback control and achieves a robust performance with respect to parameter perturbation.

Figures (13)

• Figure 1: Feedback controls of the LQ optimal liquidation problem with $Q_0\sim\mathcal{U}(1,2)$ obtained by using different methods.
• Figure 2: Absolute performance gaps of precomputed feedback controls from the FIPDE scheme (left) and the EMReg scheme (right) on perturbed models with $Q_0\sim \mathcal{U}(Q_{0,\min},Q_{0,\max})$.
• Figure 3: Relative performance gaps of precomputed feedback controls on perturbed models with $Q_0\sim \mathcal{U}(Q_{0,\min},Q_{0,\max})$; from left to right: results for the FIPDE and EMReg scheme; from top to bottom: distributions of relative errors for fixed $Q_{0,\min}$ and varying $Q_{0,\max}\in [2,2.5]$ and $Q_{0,\max}\in [1.5,2]$, where on each box, the central line is the median, the edges of the box are the 25th and 75th percentiles, the whiskers extend to non-outliers extreme data points, and outliers are plotted individually.
• Figure 4: Numerical results of the FIPDE scheme for the nonsmooth optimal liquidation problem with $k_2=1$; from left to right: approximate feedback control for $Q_0\sim \mathcal{U}(1,2)$, and absolute performance gaps of the precomputed feedback control on perturbed models with $Q_0\sim \mathcal{U}(Q_{0,\min},Q_{0,\max})$.
• Figure 5: Numerical results of the FIPDE scheme for two-dimensional C-S model with $\beta=0$; from left to right: convergence of the FIPDE method in terms of NAG iterations, and the approximate feedback control at $t=0$.

Theorems & Definitions (2)

• Remark 2.1
• Remark 2.2