Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A framework of discontinuous Galerkin neural networks for iteratively approximating residuals

Long Yuan, Hongxing Rui

TL;DR

An abstract discontinuous Galerkin neural network (DGNN) framework for analyzing the convergence of least-squares methods based on the residual minimization when feasible solutions are neural networks is proposed and a template based on the considered models for initializing nonlinear weights is designed.

Abstract

We propose an abstract discontinuous Galerkin neural network (DGNN) framework for analyzing the convergence of least-squares methods based on the residual minimization when feasible solutions are neural networks. Within this framework, we define a quadratic loss functional as in the least square method with $h-$refinement and introduce new discretization sets spanned by element-wise neural network functions. The desired neural network approximate solution is recursively supplemented by solving a sequence of quasi-minimization problems associated with the underlying loss functionals and the adaptively augmented discontinuous neural network sets without the assumption on the boundedness of the neural network parameters. We further propose a discontinuous Galerkin Trefftz neural network discretization (DGTNN) only with a single hidden layer to reduce the computational costs. Moreover, we design a template based on the considered models for initializing nonlinear weights. Numerical experiments confirm that compared to existing PINN algorithms, the proposed DGNN method with one or two hidden layers is able to improve the relative $L^2$ error by at least one order of magnitude at low computational costs.

A framework of discontinuous Galerkin neural networks for iteratively approximating residuals

TL;DR

An abstract discontinuous Galerkin neural network (DGNN) framework for analyzing the convergence of least-squares methods based on the residual minimization when feasible solutions are neural networks is proposed and a template based on the considered models for initializing nonlinear weights is designed.

Abstract

We propose an abstract discontinuous Galerkin neural network (DGNN) framework for analyzing the convergence of least-squares methods based on the residual minimization when feasible solutions are neural networks. Within this framework, we define a quadratic loss functional as in the least square method with refinement and introduce new discretization sets spanned by element-wise neural network functions. The desired neural network approximate solution is recursively supplemented by solving a sequence of quasi-minimization problems associated with the underlying loss functionals and the adaptively augmented discontinuous neural network sets without the assumption on the boundedness of the neural network parameters. We further propose a discontinuous Galerkin Trefftz neural network discretization (DGTNN) only with a single hidden layer to reduce the computational costs. Moreover, we design a template based on the considered models for initializing nonlinear weights. Numerical experiments confirm that compared to existing PINN algorithms, the proposed DGNN method with one or two hidden layers is able to improve the relative error by at least one order of magnitude at low computational costs.

Paper Structure

This paper contains 27 sections, 4 theorems, 106 equations, 12 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Mathematical setup, linear models and discontinuous neural networks
  3. Discontinuous neural networks
  4. Minimization of a new relaxed loss functional and associated variational formulation
  5. A discontinuous Galerkin neural network iterative method
  6. An overview of the DGNN method
  7. The convergence analysis
  8. Discussions on the default initial value $u_0=0$
  9. Possion equations
  10. Helmholtz equations
  11. A high-accuracy DGTNN method with a single hidden layer for PDEs with constant coefficients
  12. An effective strategy for computing the initial value $u_0$
  13. A discontinuous Galerkin Trefftz neural network method
  14. Helmholtz equations with constant coefficient $\rho=1$
  15. A high-accuracy DGNN method with two hidden layers for PDEs with variable coefficients
  16. ...and 12 more sections

Key Result

Theorem 2.1

Suppose that $s\geq 2$, and that $Q\subset \mathbb{R}^{d+1}$ is compact, then $V_n({\mathcal{T}}_h)$ is dense in the Sobolev space

Figures (12)

  • Figure 1: A schematic for the discontinuous Galerkin Neural network method with a single hidden layer.
  • Figure 2: Displacement of a string (Possion equation in two dimension). (Left) Relative error in the $L^2$-norm at each Galerkin iteration. (Right) Relative error in the broken $H^1$-seminorm at each Galerkin iteration.
  • Figure 3: Displacement of a string (Helmholtz equation in two dimension). (Left) Relative error in the $L^2$-norm at each Galerkin iteration. (Right) Relative error in the broken $H^1$-seminorm at each Galerkin iteration.
  • Figure 4: Displacement of a string (Helmholtz equation in two dimension). (Left-Up) Relative error in the $L^2$-norm at each Galerkin iteration. (Right-Up) The progress of the relative error in the $L^2$-norm within each Galerkin iteration. The x-axis thus denotes the cumulative training epoch over all Galerkin iterations. (Left-Bottom) Relative error in the broken $H^1$-seminorm at each Galerkin iteration. (Right-Bottom) The progress of the relative error in the broken $H^1$-seminorm within each Galerkin iteration.
  • Figure 5: A schematic for the discontinuous Galerkin Neural network method with two hidden layers.
  • ...and 7 more figures

Theorems & Definitions (13)

  • Theorem 2.1
  • Theorem 3.1
  • Remark 3.1
  • Remark 4.1
  • Remark 4.2
  • Theorem 4.1
  • Remark 4.3
  • Remark 4.4
  • Remark 4.5
  • Remark 4.6
  • ...and 3 more