Space-time deep neural network approximations for high-dimensional partial differential equations

TL;DR

The paper addresses the challenge of solving high-dimensional PDEs by overcoming the curse of dimensionality through a space-time deep ANN framework that leverages probabilistic representations and Monte Carlo Euler approximations. It develops rigorous SDE analyses (moment bounds and weak/strong Euler–Maruyama error estimates), connects these to PDE solutions via the Feynman–Kac formula, and then builds neural network realizations capable of approximating the full space-time PDE solution on $[0,T]\times[a,b]^d$ with polynomial-in-$d$ and polynomial-in-$\varepsilon^{-1}$ complexity. The main contributions include (i) precise a priori bounds for Gaussian increments and SDE solutions, (ii) explicit weak and strong convergence results for Euler-type schemes, (iii) Monte Carlo PDE error bounds, (iv) a comprehensive ANN framework with realizations, compositions, and space-time network constructions that yield existence results and explicit cost bounds for approximating Kolmogorov PDEs without the curse of dimensionality. Together, these results provide a rigorous theoretical foundation for scalable, deep learning-based numerical solvers for high-dimensional PDEs with provable accuracy and complexity guarantees, enabling practical applicability in high-dimensional settings.

Abstract

It is one of the most challenging issues in applied mathematics to approximately solve high-dimensional partial differential equations (PDEs) and most of the numerical approximation methods for PDEs in the scientific literature suffer from the so-called curse of dimensionality in the sense that the number of computational operations employed in the corresponding approximation scheme to obtain an approximation precision $\varepsilon>0$ grows exponentially in the PDE dimension and/or the reciprocal of $\varepsilon$. Recently, certain deep learning based approximation methods for PDEs have been proposed and various numerical simulations for such methods suggest that deep neural network (DNN) approximations might have the capacity to indeed overcome the curse of dimensionality in the sense that the number of real parameters used to describe the approximating DNNs grows at most polynomially in both the PDE dimension $d\in\mathbb{N}$ and the reciprocal of the prescribed accuracy $\varepsilon>0$. There are now also a few rigorous results in the scientific literature which substantiate this conjecture by proving that DNNs overcome the curse of dimensionality in approximating solutions of PDEs. Each of these results establishes that DNNs overcome the curse of dimensionality in approximating suitable PDE solutions at a fixed time point $T>0$ and on a compact cube $[a,b]^d$ in space but none of these results provides an answer to the question whether the entire PDE solution on $[0,T]\times [a,b]^d$ can be approximated by DNNs without the curse of dimensionality. It is precisely the subject of this article to overcome this issue. More specifically, the main result of this work in particular proves for every $a\in\mathbb{R}$, $ b\in (a,\infty)$ that solutions of certain Kolmogorov PDEs can be approximated by DNNs on the space-time region $[0,T]\times [a,b]^d$ without the curse of dimensionality.

Figures (2)

• Figure 1: Graphical illustration for the realization function and the architecture of an ANN $\Phi= ((W_1, B_1), \ldots, (W_L, B_L)) \in ( \times_{k = 1}^L(\mathbb{R}^{l_k \times l_{k-1}} \times \mathbb{R}^{l_k})) \subseteq \mathbf{N}$ (see \ref{['thm:DNNerrorEstimateLaplace']}) where $L\in\mathbb{N}$ describes the number of affine linear transformations, where $l_0,l_1,\ldots, l_L \in \mathbb{N}$ describe the dimensions of the layers of the ANN, and where the function $A \colon (\cup_{d \in \mathbb{N}} \mathbb{R}^d) \to (\cup_{d \in \mathbb{N}} \mathbb{R}^d)$ represents the activation function (see \ref{['eq:intro:activ']}).
• Figure 2: Graphical illustration for the realization function and the architecture of an ANN $\Phi= ((W_1, B_1), \ldots, (W_L, B_L)) \in ( \times_{k = 1}^L(\mathbb{R}^{l_k \times l_{k-1}} \times \mathbb{R}^{l_k})) \subseteq \mathbf{N}$ (see \ref{['Def:ANN']}) where $L\in\mathbb{N}$ describes the number of affine linear transformations, where $l_0,l_1,\ldots, l_L \in \mathbb{N}$ describe the dimensions of the layers of the ANN, and where the function $a \colon \mathbb{R} \to \mathbb{R}$ represents the activation function (see \ref{['Definition:ANNrealization']}).

Theorems & Definitions (37)

• Theorem 1.1
• Example 1.2
• Definition 2.1: Standard norms
• Lemma 2.2
• Lemma 2.3
• Proposition 2.4
• proof : Proof of Proposition \ref{['prop:perturbation_PDE_2']}
• Lemma 2.5
• proof : Proof of Lemma \ref{['lem:Euler']}
• Proposition 3.1