Solving High Dimensional Partial Differential Equations Using Tensor Neural Network and A Posteriori Error Estimators

TL;DR

This work addresses solving high-dimensional elliptic PDEs and eigenvalue problems by marrying tensor neural networks (TNNs) with a posteriori error estimators to form adaptive, mesh-free loss functions. The key idea is to use the estimator to bound the energy error and drive adaptive selection of a p-term subspace, while exploiting the TNN's separable structure to perform accurate, low-cost high-dimensional quadrature. The authors develop explicit estimators for homogeneous/non-homogeneous Dirichlet, Neumann, and eigenvalue problems, construct corresponding loss functions, and provide Galerkin-based training algorithms that alternately update linear coefficients and neural-network parameters. Numerical experiments in dimensions up to 20 demonstrate high accuracy and favorable computation times, and the approach avoids balancing boundary-domain losses typical in other NN methods, with potential impact on scalable PDE solvers in physics and engineering.

Abstract

In this paper, based on the combination of tensor neural network and a posteriori error estimator, a novel type of machine learning method is proposed to solve high-dimensional boundary value problems with homogeneous and non-homogeneous Dirichlet or Neumann type of boundary conditions and eigenvalue problems of the second-order elliptic operator. The most important advantage of the tensor neural network is that the high dimensional integrations of tensor neural networks can be computed with high accuracy and high efficiency. Based on this advantage and the theory of a posteriori error estimation, the a posteriori error estimator is adopted to design the loss function to optimize the network parameters adaptively. The applications of tensor neural network and the a posteriori error estimator improve the accuracy of the corresponding machine learning method. The theoretical analysis and numerical examples are provided to validate the proposed methods.

Figures (5)

• Figure 1: Architecture of TNN. Black arrows mean linear transformation (or affine transformation). Each ending node of blue arrows is obtained by taking the scalar multiplication of all starting nodes of blue arrows that end in this ending node. The final output of TNN is derived from the summation of all starting nodes of red arrows.
• Figure 2: Relative errors during the training process for the homogeneous Dirichlet boundary problem: $d=5$, $10$, and $20$. The upper row shows the relative $L^2(\Omega)$ errors, and the down row shows the relative $H^1(\Omega)$ errors of solution approximations.
• Figure 3: Relative errors during the training process for non-homogeneous Neumann boundary problems: $d=5$, $10$, and $20$. The upper row shows the relative $L^2(\Omega)$ errors and the down row shows the relative $H^1(\Omega)$ errors of solution approximations.
• Figure 4: Relative errors during the training process for Laplace eigenvalue problem: $d=5$, $10$, and $20$. The left column shows the relative errors of eigenvalue approximations, the middle column shows the relative $L^2(\Omega)$ errors and the right column shows the relative $H^1(\Omega)$ errors of eigenfunction approximations.
• Figure 5: Relative errors during the L-BFGS training process for harmonic oscillator eigenvalue problem: $d=5$, $10$, and $20$. The left column shows the relative errors of eigenvalue approximations, the middle column shows the relative $L^2(\Omega)$ errors and the right column shows the relative $H^1(\Omega)$ errors of eigenfunction approximations.

Theorems & Definitions (14)

• Theorem 2.1
• Theorem 2.2
• Remark 2.1
• Lemma 3.1
• Theorem 3.1
• proof
• Theorem 3.2
• proof
• Theorem 3.3
• proof