Leveraging the turnpike effect for Mean Field Games numerics

TL;DR

A “turnpike-accelerated” version of DGM for finite horizon Mean Field Games is introduced that incorporates the turnpike estimates in the loss function to be optimized, and a comparative numerical analysis is performed to show the advantages of this accelerated version over the baseline DGM algorithm.

Abstract

Recently, a deep-learning algorithm referred to as Deep Galerkin Method (DGM), has gained a lot of attention among those trying to solve numerically Mean Field Games with finite horizon, even if the performance seems to be decreasing significantly with increasing horizon. On the other hand, it has been proven that some specific classes of Mean Field Games enjoy some form of the turnpike property identified over seven decades ago by economists. The gist of this phenomenon is a proof that the solution of an optimal control problem over a long time interval spends most of its time near the stationary solution of the ergodic solution of the corresponding infinite horizon optimization problem. After reviewing the implementation of DGM for finite horizon Mean Field Games, we introduce a ``turnpike-accelerated'' version that incorporates the turnpike estimates in the loss function to be optimized, and we perform a comparative numerical analysis to show the advantages of this accelerated version over the baseline DGM algorithm. We demonstrate on some of the Mean Field Game models with local-couplings known to have the turnpike property, as well as a new class of linear-quadratic models for which we derive explicit turnpike estimates.

Figures (12)

• Figure 1: Behavior of the solution $(u^T; m^T)$ of the finite horizon game
• Figure 2: Behavior of the solution $(u^T; m^T)$ of the finite horizon game
• Figure 3: Norm of $m^T - \overline{m}$ and $u^T-\langle u^T \rangle - \overline{u}$
• Figure 4: Relative absolute error for the density (on the left) and the value function (on the right). The axes correspond to the index on a uniform grid of size 200 for time in $[0,T]$ and space in $[0,1]$.
• Figure 5: Relative absolute error for the density (left) and the value function (right) with respect to the training iteration index.

Theorems & Definitions (7)

• Proposition 2.1: Proposition 1 in porretta_long_2012
• Theorem 2.1: Theorem 2.1 in porretta_long_2012
• Proposition 2.2: Lemma 2.4 and (13) in porretta_long_2012
• Theorem 2.2
• Theorem 4.1
• Proposition 4.1
• proof