Improved physics-informed neural network in mitigating gradient related failures

TL;DR

This work tackles gradient-flow stiffness in physics-informed neural networks (PINNs) by proposing an Improved PINN (I-PINN) that fuses an enhanced neural-architecture with adaptive loss weighting bounded by an upper limit γ. Building on IA-PINN and IAW-PINN, I-PINN enforces balanced training between supervised and unsupervised components through a capped uncertainty-based weighting scheme, and an improved multilayer perceptron to improve gradient flow. Empirical results on Helmholtz, Klein–Gordon, and lid-driven cavity problems demonstrate at least an order-of-magnitude improvement in accuracy over PINN, IA-PINN, and IAW-PINN, with robust convergence across network configurations. The approach maintains baseline computational complexity while enhancing stability and generalization, and code is publicly available for reproducibility and broader adoption.

Abstract

Physics-informed neural networks (PINNs) integrate fundamental physical principles with advanced data-driven techniques, driving significant advancements in scientific computing. However, PINNs face persistent challenges with stiffness in gradient flow, which limits their predictive capabilities. This paper presents an improved PINN (I-PINN) to mitigate gradient-related failures. The core of I-PINN is to combine the respective strengths of neural networks with an improved architecture and adaptive weights containingupper bounds. The capability to enhance accuracy by at least one order of magnitude and accelerate convergence, without introducing extra computational complexity relative to the baseline model, is achieved by I-PINN. Numerical experiments with a variety of benchmarks illustrate the improved accuracy and generalization of I-PINN. The supporting data and code are accessible at https://github.com/PanChengN/I-PINN.git, enabling broader research engagement.

Figures (13)

• Figure 1: Comparison of the relative errors in solving $u$ using different network structures between the AW-PINN and the IAW-PINN with upper bound $\gamma = 10^3$
• Figure 2: Comparison of the relative errors in solving $f$ using different network structures between the AW-PINN and the IAW-PINN with upper bound $\gamma = 10^3$
• Figure 3: Comparison of the relative errors in solving $u$ and $f$ using different network structures between the AW-PINN and the IAW-PINN with upper bound $\gamma = 10^3$
• Figure 4: The figure shows the progression of loss function weights over 40,000 iterations of network optimization using Adam's optimizer, within a hidden layer network structure of $7 \times 50$, employing the AW-PINN.
• Figure 5: Variation of loss function weights obtained by optimizing the Network 40,000 times with Adam's optimizer under a hidden layer network structure $7 \times 50$ using IAW-PINN with upper bound $\gamma = 10^3$