From Monte Carlo to neural networks approximations of boundary value problems

TL;DR

The paper tackles solving the Poisson boundary value problem on bounded domains with Hölder data by marrying a probabilistic Monte Carlo approach, based on a modified Walk-on-Spheres acceleration, with a constructive ReLU deep neural network framework. It delivers uniform-in-$x$ sup-norm error control, tail guarantees, and poly(dim) complexity for both the Monte Carlo solver and the neural surrogates, while handling general domain geometries via exterior-ball and annular-diameter geometry notions. The DNN contributions are explicit and random, enabling operator-learning-like behavior that generalizes across data $(f,g)$ with a library-like composition of neural networks for data and the solution operator. Numerical experiments up to dimension $d=100$ corroborate the theoretical scaling and demonstrate practical GPU-accelerated performance, suggesting a viable path for high-dimensional PDE solvers that break the curse of dimensionality.

Abstract

In this paper we study probabilistic and neural network approximations for solutions to Poisson equation subject to Holder data in general bounded domains of $\mathbb{R}^d$. We aim at two fundamental goals. The first, and the most important, we show that the solution to Poisson equation can be numerically approximated in the sup-norm by Monte Carlo methods, and that this can be done highly efficiently if we use a modified version of the walk on spheres algorithm as an acceleration method. This provides estimates which are efficient with respect to the prescribed approximation error and with polynomial complexity in the dimension and the reciprocal of the error. A crucial feature is that the overall number of samples does not not depend on the point at which the approximation is performed. As a second goal, we show that the obtained Monte Carlo solver renders in a constructive way ReLU deep neural network (DNN) solutions to Poisson problem, whose sizes depend at most polynomialy in the dimension $d$ and in the desired error. In fact we show that the random DNN provides with high probability a small approximation error and low polynomial complexity in the dimension.

Figures (8)

• Figure 1: The evolution of (Monte Carlo estimates of) $\mathbb{E}\left\{1/|D|\int_D \left | u(x)-u_M^{N}(x)\right| \; dx\right\}$ and $\mathbb{E}\left\{\sup_{x\in D} \left | u(x)-u_M^{N}(x)\right|\right\}$ w.r.t. $N$, for $D=D_{\sf ac}$ (see \ref{['S:numerics']}), $d=100$, $M=500$. The computed errors are decreasing to a small value as the number of WoS trajectories $N$ increases. The limit error attained when $N$ goes to infinity is not zero as it depends on $M$, but it decreases to zero as the latter parameter is also increased to infinity; see \ref{['S:numerics']} for more details.
• Figure 2: $D_{\sf c}$ and $D_{\sf ac}$ for $d=2$
• Figure 3: ${\sf U}_{\sf bound}(d,\cdot,x_0,\varepsilon)$ (red) vs $\mathbb{P}_N(d,\cdot,x_0,\varepsilon)$ (black) for $D=D_{\sf c}$, $d=20,\varepsilon=10^{-3}, N=5\times 10^4$, whilst $x_0$ is arbitrarily chosen in $D$ such that $|x_0|=0.5$.
• Figure 4: ${\sf U}_{\sf bound}(\cdot,\cdot,x_0,\varepsilon)$ (red) vs $\mathbb{P}_N(\cdot,\cdot,x_0,\varepsilon)$ (black) for $D=D_{\sf c}$, $\varepsilon=10^{-3}, N=5\times 10^4$, whilst $x_0$ is arbitrarily chosen in $D$ such that $|x_0|=0.5$, for each dimension $d$.
• Figure 5: ${\sf U}_{\sf bound}(\cdot,\cdot,x_0,\varepsilon)$ (red) vs $\mathbb{P}_N(\cdot,\cdot,x_0,\varepsilon)$ (black) for $D=D_{\sf ac}$, $\varepsilon=10^{-3}$, $N=5\times 10^4$, whilst $x_0$ is arbitrarily chosen in $D$ such that $|x_0|=0.7$, for each dimension $d$.

Theorems & Definitions (72)

• Theorem : Part I; see \ref{['thm:main']} for the full quantitative version
• Remark 1.1
• Theorem : Part II; see Theorem \ref{['thm:mainNN']} for details
• Lemma 2.1
• Theorem 2.2: Ge92
• Corollary 2.3
• Proposition 2.4
• Proposition 2.5
• Proposition 2.6
• Definition 2.7