Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Review of Neural Network Solvers for Second-order Boundary Value Problems

Ramesh Chandra Sau, Luowei Yin

TL;DR

The paper surveys neural-network solvers for second-order boundary value problems, focusing on PINN, DRM, VPINN, and WAN, and analyzes how loss formulations and optimization strategies impact performance. It synthesizes numerical results across low- and high-dimensional problems, showing that weak-form methods (DRM, VPINN) better handle low-regularity solutions than PINN, while WAN can excel on weak solutions but suffers from training instability. An error analysis for PINN is presented, decomposing the total error into approximation, statistical, and optimization components and providing a framework to bound the solution error in terms of these factors. The work highlights practical considerations for method selection, such as dimensionality, solution regularity, and training stability, and outlines theoretical gaps in optimization analysis. Overall, it offers guidance for practitioners on selecting and tuning neural solvers for PDEs and motivates further theoretical development of convergence guarantees and stability.

Abstract

Deep learning-based partial differential equation(PDE) solvers have received much attention in the past few years. Methods of this category can solve a wide range of PDEs with high accuracy, typically by transforming the problems into highly nonlinear optimization problems of neural network parameters. This work reviews several deep learning solvers proposed a few years ago, including PINN, WAN, DRM, and VPINN. Numerical results are provided to make comparisons amongst them and address the importance of loss formulation and the optimization method. A rigorous error analysis for PINN is also presented. Finally, we discuss the current limitations and bottlenecks of these methods.

A Review of Neural Network Solvers for Second-order Boundary Value Problems

TL;DR

The paper surveys neural-network solvers for second-order boundary value problems, focusing on PINN, DRM, VPINN, and WAN, and analyzes how loss formulations and optimization strategies impact performance. It synthesizes numerical results across low- and high-dimensional problems, showing that weak-form methods (DRM, VPINN) better handle low-regularity solutions than PINN, while WAN can excel on weak solutions but suffers from training instability. An error analysis for PINN is presented, decomposing the total error into approximation, statistical, and optimization components and providing a framework to bound the solution error in terms of these factors. The work highlights practical considerations for method selection, such as dimensionality, solution regularity, and training stability, and outlines theoretical gaps in optimization analysis. Overall, it offers guidance for practitioners on selecting and tuning neural solvers for PDEs and motivates further theoretical development of convergence guarantees and stability.

Abstract

Deep learning-based partial differential equation(PDE) solvers have received much attention in the past few years. Methods of this category can solve a wide range of PDEs with high accuracy, typically by transforming the problems into highly nonlinear optimization problems of neural network parameters. This work reviews several deep learning solvers proposed a few years ago, including PINN, WAN, DRM, and VPINN. Numerical results are provided to make comparisons amongst them and address the importance of loss formulation and the optimization method. A rigorous error analysis for PINN is also presented. Finally, we discuss the current limitations and bottlenecks of these methods.
Paper Structure (15 sections, 10 theorems, 71 equations, 10 figures, 3 algorithms)

This paper contains 15 sections, 10 theorems, 71 equations, 10 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Deep neural networks
  3. PDE solvers by deep neural networks
  4. Physics informed neural networks(PINNs)
  5. Deep Ritz method(DRM)
  6. Variational PINN (VPINN)
  7. Weak adversarial network(WAN)
  8. Numerical Experiments
  9. Low dimension examples
  10. Training instability of WAN
  11. High dimension examples
  12. Error analysis for PINNs method
  13. Approximation Error
  14. Statistical errors
  15. Acknowledgements

Key Result

Lemma 5.1

Let $u$ be the classical solution of eqn:Poisson. Let $u_{\theta}\in \mathcal{U}$, then the following estimate holds

Figures (10)

  • Figure 1: A four-layer neural network
  • Figure 2: (a) Exact solution($u$) (b) Neural network approximate solution($u_{\theta}$) (c) pointwise error $|u_{\theta}(x)- u(x)|$
  • Figure 3: Loss history($\hat{\mathcal{L}}(u_{\theta})$), for Example \ref{['example:pinn2d']}
  • Figure 4: Approximate solutions in the top row and pointwise errors $|u_{\theta}(x)- u(x)|$ in the bottom row for Example \ref{['example:Ldomain']}.
  • Figure 5: Approximate solutions in the top row and pointwise errors $|u_{\theta}(x)- u(x)|$ in the bottom row for Example \ref{['example:weak']}.
  • ...and 5 more figures

Theorems & Definitions (27)

  • Remark 3.1
  • Example 4.1: PINN in 2D
  • Example 4.2: DRM and VPINN in 2D
  • Example 4.3: WAN in 2D
  • Example 4.4: PINN
  • Example 4.5
  • Lemma 5.1
  • proof
  • Theorem 5.1
  • proof
  • ...and 17 more