Transformed Physics-Informed Neural Networks for The Convection-Diffusion Equation

TL;DR

This work tackles numerical difficulties in singularly perturbed convection-diffusion problems, where steep boundary layers hinder traditional solvers. It proposes using Physics-Informed Neural Networks as corrective surrogates, applied both to oscillatory FDM solutions and to reduced hyperbolic-limit solutions, and introduces linear input transformations to align boundary layers with the network's activation gradients. Neural Tangent Kernel analysis is employed to explain how input transformations change convergence and the location of learned boundary layers, supported by 1D and 2D demonstrations. The results show that appropriately chosen input shifts or shifts with scaling can dramatically improve accuracy and training efficiency, enabling robust PINN-based corrections beyond standard regimes and suggesting wider applicability to other singular perturbation problems.

Abstract

Singularly perturbed problems are known to have solutions with steep boundary layers that are hard to resolve numerically. Traditional numerical methods, such as Finite Difference Methods (FDMs), require a refined mesh to obtain stable and accurate solutions. As Physics-Informed Neural Networks (PINNs) have been shown to successfully approximate solutions to differential equations from various fields, it is natural to examine their performance on singularly perturbed problems. The convection-diffusion equation is a representative example of such a class of problems, and we consider the use of PINNs to produce numerical solutions of this equation. We study two ways to use PINNS: as a method for correcting oscillatory discrete solutions obtained using FDMs, and as a method for modifying reduced solutions of unperturbed problems. For both methods, we also examine the use of input transformation to enhance accuracy, and we explain the behavior of input transformations analytically, with the help of neural tangent kernels.

Figures (14)

• Figure 1: Image of a fully-connected neural network.
• Figure 2: Exact solutions of Equation \ref{['eq:1dcd2']} for $\epsilon\in \{0.1,0.01,0.001,0.0001\}$.
• Figure 3: Corrections made by standard PINN on FDM oscillatory solutions for: $\epsilon=0.01$ in Figure \ref{['fig:1dcdd1e-2']} and $\epsilon=0.001$ in Figure \ref{['fig:1dcdd1e-3']}.
• Figure 4: Corrections made by PINN with input transformation $T(x) = 100(x-1)$ on FDM oscillatory solutions for: $\epsilon=0.01$ in Figure \ref{['fig:1dcdd1e-2_trans']} and $\epsilon=0.001$ Figure \ref{['fig:1dcdd1e-3_trans']}.
• Figure 5: Different approximations generated using a standard PINNs under different random initialization seeds in \ref{['fig:seeds']}. Comparison of effects of transformation in \ref{['fig:ib']}. Here the input is transformed from $x\in [0,1]$ to $x-1\in [-1,0]$.