On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs

TL;DR

This work provides a theoretical foundation for the convergence of Physics-Informed Neural Networks (PINNs) when solving linear second-order elliptic and parabolic PDEs. By establishing a Hölder-regularized empirical loss framework and leveraging the Schauder approach, the authors prove that minimizers converge to the PDE solution in $C^0$ (and in $H^1$ under suitable boundary conditions) with probability 1 as data increase. The results are complemented by computational examples (Poisson and heat equations) that illustrate convergence behavior and demonstrate the practical impact of regularization on generalization. Overall, the paper delivers the first rigorous consistency results for PINNs in the sampled data limit for these PDE classes and clarifies how boundary data influence convergence rates.

Abstract

Physics informed neural networks (PINNs) are deep learning based techniques for solving partial differential equations (PDEs) encounted in computational science and engineering. Guided by data and physical laws, PINNs find a neural network that approximates the solution to a system of PDEs. Such a neural network is obtained by minimizing a loss function in which any prior knowledge of PDEs and data are encoded. Despite its remarkable empirical success in one, two or three dimensional problems, there is little theoretical justification for PINNs. As the number of data grows, PINNs generate a sequence of minimizers which correspond to a sequence of neural networks. We want to answer the question: Does the sequence of minimizers converge to the solution to the PDE? We consider two classes of PDEs: linear second-order elliptic and parabolic. By adapting the Schauder approach and the maximum principle, we show that the sequence of minimizers strongly converges to the PDE solution in $C^0$. Furthermore, we show that if each minimizer satisfies the initial/boundary conditions, the convergence mode becomes $H^1$. Computational examples are provided to illustrate our theoretical findings. To the best of our knowledge, this is the first theoretical work that shows the consistency of PINNs.

Figures (5)

• Figure 1: Schematic of physics-informed neural networks (PINNs). The left part visualizes a standard neural network parameterized by $\theta$. The right part applies the given physical laws to the network. $\mathcal{L}$ and $\mathcal{B}$ are the differential and the boundary operators, respectively. The PDE data ($f, g$) are obtained from random sample points. The loss function is computed by evaluating $\mathcal{L}[u]$ and $\mathcal{B}[u]$ on the sample points, which can be done efficiently through automatic differentiation Baydin_17AD. Minimizing the loss with respect to the network's parameters $\theta$ produces a PINN $u(x;\theta^*)$, which serves as an approximation to the solution to the PDE.
• Figure 2: Illustration of the total errors. $\mathcal{H}_n$ is the chosen function class. $u^*$ is the solution to the underlying PDE. The number of training data is $m$. $h_m$ is a minimizer of the loss with $m$ data. $\hat{h}$ is a function in $\mathcal{H}_n$ that minimizes the loss with infinitely many data. $\tilde{h}_m$ is an approximation that one obtains in practice, e.g., the result obtained after 1M epochs of a gradient-based optimization.
• Figure 3: The $L^2$ and $H^1$ convergence for the 1D Poisson equation whose exact solution is $u^*(x)=\tanh(x)$ with respect to the number of training data points. The residual neural networks of depth 2 and width 50 are employed. The 'PINN' results are shown as dash-dot lines and the 'LIPR' results are shown as solid lines. The dotted line is a reference line indicating the $\mathcal{O}(m_r^{-1})$-rate of convergence.
• Figure 4: The $L^2$ and $H^1$ convergence for the 1D Poisson equation whose exact solution is $u^*(x) = (1-x^2)\sin(6\pi x)$ with respect to the number of training data points. The neural networks \ref{['def:bdry-match-NNs']} that automatically satisfy the boundary conditions are employed. The 'PINN' results are shown as dash-dot lines and the 'LIPR' results are shown as solid lines. The dotted line is a reference line indicating the $\mathcal{O}(m_r^{-1})$-rate of convergence.
• Figure 5: The $L^2(0,1;L^2)$- and $L^2(0,1;H^1)$-convergence of the errors with respect to the number of training data point for the 1D Heat equation whose exact solution is $u^*(x) = \sin(\pi x)e^{-t}$. The feed-forward neural networks of depth 2 and width 50 are employed. The 'PINN' results are shown as dash-dot lines and the 'LIPR' results are shown as solid lines. The dotted lines are reference lines indicating the $\mathcal{O}(m_r^{-\frac{1}{2}})$- and $\mathcal{O}(m_r^{-1})$-rate of convergence.

Theorems & Definitions (43)

• Remark 2.1
• Definition 2.1
• Definition 2.2
• Lemma 2.1
• proof
• Remark 2.2
• Theorem 3.1
• proof
• Remark 3.1
• Theorem 3.2