A deep learning method for solving stochastic optimal control problems driven by fully-coupled FBSDEs

TL;DR

This work addresses numerical solution of high-dimensional stochastic optimal control problems driven by fully-coupled FBSDEs by reformulating the problem as a stochastic Stackelberg differential game. It introduces a bi-level optimization scheme that alternates updates between a follower network (y0,z(·)) minimizing J(y0,z(·)) and a leader network (u(·)) minimizing J(u(·)), with time-discretized dynamics via Euler-Maruyama and Monte Carlo estimation. Three neural networks are employed: NN^θ_u for the control, NN^{θ_y} for the initial y0, and NN^{θ_z} for the z(t) process, all sharing a time-extended input and using batch normalization; training enforces κ inner updates for the follower before a single update for the leader. Numerical experiments on investment-consumption problems with stochastic recursive utilities demonstrate that the method yields follower and leader objectives close to classical solutions in the linear case and remains effective for nonlinear drivers, highlighting scalability to higher dimensions and complex dynamics.

Abstract

In this paper,we mainly focus on the numerical solution of high-dimensional stochastic optimal control problem driven by fully-coupled forward-backward stochastic differential equations (FBSDEs in short) through deep learning. We first transform the problem into a stochastic Stackelberg differential game problem (leader-follower problem), then a bi-level optimization method is developed where the leader's cost functional and the follower's cost functional are optimized alternatively via deep neural networks. As for the numerical results, we compute two examples of the investment-consumption problem solved through stochastic recursive utility models, and the results of both examples demonstrate the effectiveness of our proposed algorithm.

Figures (6)

• Figure 1: Representation of a single neural network with $I=4$, $d_1=d_2=d_3=4$, $n=2$, $d_4=k=1$.
• Figure 2: For easier notation, here we use $x_i$ to represent $x_{t_i}$, and $y_i$, $z_i$, $u_i$ for $y_{t_i}$, $z_{t_i}$, $u_{t_i}$ respectively. In the whole neural network structure, $h_y$, $h_z$ and $h_u$ represent the hidden layers of $\mathcal{NN}^{\theta_y}$, $\mathcal{NN}^{\theta_z}$ and $\mathcal{NN}^{\theta_u}$, respectively. And a common neural network is used for all the discrete time points.
• Figure 3: Case $n=50$ and $T=1.0$
• Figure 4: The distance of $y_0$ between the integral form and the parametric form and the values of $y_0$ with different $\kappa$ for $n=10$ and $T=0.5$.
• Figure 5: Case $n=50$ and $T=0.5$. The left figure shows the mean and scope for the values of $y_0$ among 5 independent runs. We can see that the two different forms of $y_0$ are getting closer when the number of iteration steps increases. Moreover, the value of the integral form of $y_0$ (see the green curve and scope) is getting larger and then tends to be stable. The right figure shows the mean and scope of the distances between the different forms of $y_0$.

Theorems & Definitions (2)

• Lemma 1
• Remark