Adaptive Basis-inspired Deep Neural Network for Solving Partial Differential Equations with Localized Features

TL;DR

The paper tackles PDEs with localized features by introducing Basis-inspired DNNs (BI-DNN) built from Basis-inspired Blocks that mimic FEM basis functions and leveraging the Kolmogorov Superposition Theorem to handle high dimensions. An AFEM-inspired adaptive framework, ABI-DNN, then augments BI-DNNs by adding BI-blocks in regions of high estimated error, retraining until a prescribed tolerance is reached. Empirical results on function fitting, Poisson problems with singularities, and Burgers equation show that BI-DNNs outperform PINNs at similar parameter counts, and ABI-DNN delivers further accuracy gains by automatic architecture refinement, particularly in challenging localized regions. The approach offers a principled way to combine FEM-inspired locality with neural networks, enabling efficient and accurate PDE solvers for problems with sharp gradients and singularities.

Abstract

This paper proposes an Adaptive Basis-inspired Deep Neural Network (ABI-DNN) for solving partial differential equations with localized phenomena such as sharp gradients and singularities. Like the adaptive finite element method, ABI-DNN incorporates an iteration of "solve, estimate, mark, enhancement", which automatically identifies challenging regions and adds new neurons to enhance its capability. A key challenge is to force new neurons to focus on identified regions with limited understanding of their roles in approximation. To address this, we draw inspiration from the finite element basis function and construct the novel Basis-inspired Block (BI-block), to help understand the contribution of each block. With the help of the BI-block and the famous Kolmogorov Superposition Theorem, we first develop a novel fixed network architecture named the Basis-inspired Deep Neural Network (BI-DNN), and then integrate it into the aforementioned adaptive framework to propose the ABI-DNN. Extensive numerical experiments demonstrate that both BI-DNN and ABI-DNN can effectively capture the challenging singularities in target functions. Compared to PINN, BI-DNN attains significantly lower relative errors with a similar number of trainable parameters. When a specified tolerance is set, ABI-DNN can adaptively learn an appropriate architecture that achieves an error comparable to that of BI-DNN with the same structure.

Figures (23)

• Figure 1: Illustration of basis-inspired blocks with activation functions: (a) ReLU (Equation \ref{['eq:bi-block:relu:v2']}) and (b) Tanh (Equation \ref{['eq:bi-blocks:tanh:v2']}), respectively.
• Figure 2: Image of basis-inspired blocks with (a) ReLU and (b) Tanh activation functions, respectively. Here, equally spaced nodes within the interval $[-2, 2]$ are used to determine the weights and biases in BI-blocks following equations \ref{['eq:bi-block:relu:weight:v2']}, \ref{['eq:bi-block:relu:bias:v2']} and \ref{['eq:bi-blocks:tanh:params:v2']}.
• Figure 3: Illustration of the architecture of a one-dimensional BI-DNN consisting of three BI-blocks with the Tanh activation followed by a three-layer subnetwork. The only difference to a conventional DNN is the BI-blocks following the input layer.
• Figure 4: Illustration of the architecture of a two-dimensional BI-DNN consisting of two BI-blocks with Tanh activation function in each dimension.
• Figure 5: Comparison of relative errors between frozen BI-DNNs and BI-DNNs on Problem \ref{['pb:1d:singular']} (Top row) and Problem \ref{['pb:1d:smooth']} (Bottom row).

Theorems & Definitions (1)

• Theorem 1: Kolmogorov Superposition Theorem