Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep collocation method: A framework for solving PDEs using neural networks with error control

Mingxing Weng, Zhiping Mao, Jie Shen

TL;DR

The paper addresses the accuracy and efficiency limitations of AI-based PDE solvers by introducing a deep collocation framework that builds adaptive neural-network–based bases within a collocation least-squares setting. Solutions lie in the span of these bases, updated greedily and guided by the equation residual, with adaptive collocation and initialization strategies ensuring robustness. A formal convergence analysis yields an a posteriori error estimator and geometric convergence under suitable conditions, validated across function fitting, boundary-layer, Poisson, biharmonic, and Allen–Cahn problems. The method demonstrates high accuracy, robustness to sharp gradients and singularities, and broad applicability to nonlinear and complex-domain PDEs, with potential for extensions to higher dimensions.

Abstract

Neural networks have shown significant potential in solving partial differential equations (PDEs). While deep networks are capable of approximating complex functions, direct one-shot training often faces limitations in both accuracy and computational efficiency. To address these challenges, we propose an adaptive method that uses single-hidden-layer neural networks to construct basis functions guided by the equation residual. The approximate solution is computed within the space spanned by these basis functions, employing a collocation least squares scheme. As the approximation space gradually expands, the solution is iteratively refined; meanwhile, the progressive improvements serve as reliable {\it a posteriori} error indicators that guide the termination of the sequential updates. Additionally, we introduce adaptive strategies for collocation point selection and parameter initialization to enhance robustness and improve the expressiveness of the neural networks. We also derive the approximation error estimate and validate the proposed method with several numerical experiments on various challenging PDEs, demonstrating both high accuracy and robustness of the proposed method.

Deep collocation method: A framework for solving PDEs using neural networks with error control

TL;DR

The paper addresses the accuracy and efficiency limitations of AI-based PDE solvers by introducing a deep collocation framework that builds adaptive neural-network–based bases within a collocation least-squares setting. Solutions lie in the span of these bases, updated greedily and guided by the equation residual, with adaptive collocation and initialization strategies ensuring robustness. A formal convergence analysis yields an a posteriori error estimator and geometric convergence under suitable conditions, validated across function fitting, boundary-layer, Poisson, biharmonic, and Allen–Cahn problems. The method demonstrates high accuracy, robustness to sharp gradients and singularities, and broad applicability to nonlinear and complex-domain PDEs, with potential for extensions to higher dimensions.

Abstract

Neural networks have shown significant potential in solving partial differential equations (PDEs). While deep networks are capable of approximating complex functions, direct one-shot training often faces limitations in both accuracy and computational efficiency. To address these challenges, we propose an adaptive method that uses single-hidden-layer neural networks to construct basis functions guided by the equation residual. The approximate solution is computed within the space spanned by these basis functions, employing a collocation least squares scheme. As the approximation space gradually expands, the solution is iteratively refined; meanwhile, the progressive improvements serve as reliable {\it a posteriori} error indicators that guide the termination of the sequential updates. Additionally, we introduce adaptive strategies for collocation point selection and parameter initialization to enhance robustness and improve the expressiveness of the neural networks. We also derive the approximation error estimate and validate the proposed method with several numerical experiments on various challenging PDEs, demonstrating both high accuracy and robustness of the proposed method.

Paper Structure

This paper contains 20 sections, 4 theorems, 50 equations, 13 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem Statement and Collocation Scheme
  4. Construction of Neural Network Basis
  5. Adaptive Strategies
  6. Adaptive Collocations
  7. Adaptive Initialization
  8. Numerical analysis
  9. Numerical Results
  10. Function Fitting
  11. Advection-diffusion problem with Boundary Layer
  12. Poisson Equation
  13. L-shaped domain
  14. Rapidly Varying Source Term
  15. Fourth-order problem: Biharmonic equation
  16. ...and 5 more sections

Key Result

Proposition 3.1

\newlabelprop:converge0 Consider the linear boundary value problem equ:linear_pde with a source term $f \in L^2(\Omega)$ and boundary data $g \in L^2(\partial\Omega)$. The error defined in equ:error_variants either strictly decreases at each stage $s \in \mathbb{N}^+$, i.e., ${\left\vert\left\vert

Figures (13)

  • Figure 1: An example of biases adaptive initialization. Left: The target distribution $|\!\sin x \cos y|$; Right: The density distribution of 20000 initialized hyperplanes computed with $\tau = 0.1$.
  • Figure 1: Results for Subsection \ref{['subsec:function_fitting']}. Left: the true and a posteriori errors of stage 2, 4, and 6. Right: Error vs. # of stages.
  • Figure 2: Approximation solution and the corresponding absolute error for Subsection \ref{['subsec:boundary-layer']}.
  • Figure 3: Results for Subsection \ref{['subsec:boundary-layer']}. Left: the true and a posteriori errors of stage 2,3, and 4. Right: Error vs. # of stages.
  • Figure 4: Poisson equation on a L-shaped domain. The absolute errors obtained without (left) and with (middle) the inclusion of knowledge-based hidden neurons. Right: $L^2$ and $L^\infty$ errors computed at each stage.
  • ...and 8 more figures

Theorems & Definitions (8)

  • Proposition 3.1
  • Proof 1
  • Theorem 3.2: Universal Approximation hornik_approximation_1991
  • Proposition 3.3
  • Proof 2
  • Proposition 3.4
  • Proof 3
  • Remark 3.5