ProPINN: Demystifying Propagation Failures in Physics-Informed Neural Networks

TL;DR

Propagation failures in PINNs are shown to stem from insufficient propagation of correct supervision into interior regions, formally linked to low gradient correlation $G_{u_\theta}(\mathbf{x},\mathbf{x}')$ between nearby points. By contrasting PINN with FEM discretizations, the authors develop ProPINN, an architecture that unites region gradients through differential perturbation and multi-region mixing to boost $G_{u_\theta}$ and improve propagation. The method achieves state-of-the-art performance across standard PDE benchmarks and challenging Navier–Stokes tasks, outperforming strong Transformer-based baselines by around 46% relative on multiple tasks and with favorable efficiency. This architecture-centered solution provides a precise diagnostic criterion for PINN failures and offers a practical, scalable approach to reliable PDE solving with neural networks.

Abstract

Physics-informed neural networks (PINNs) have earned high expectations in solving partial differential equations (PDEs), but their optimization usually faces thorny challenges due to the unique derivative-dependent loss function. By analyzing the loss distribution, previous research observed the propagation failure phenomenon of PINNs, intuitively described as the correct supervision for model outputs cannot ''propagate'' from initial states or boundaries to the interior domain. Going beyond intuitive understanding, this paper provides a formal and in-depth study of propagation failure and its root cause. Based on a detailed comparison with classical finite element methods, we ascribe the failure to the conventional single-point-processing architecture of PINNs and further prove that propagation failure is essentially caused by the lower gradient correlation of PINN models on nearby collocation points. Compared to superficial loss maps, this new perspective provides a more precise quantitative criterion to identify where and why PINN fails. The theoretical finding also inspires us to present a new PINN architecture, named ProPINN, which can effectively unite the gradients of region points for better propagation. ProPINN can reliably resolve PINN failure modes and significantly surpass advanced Transformer-based models with 46% relative promotion.

Figures (13)

• Figure 1: Comparison of PINN and ProPINN on Convection ($\frac{\partial u}{\partial t}+50\frac{\partial u}{\partial x}=0$). In addition to the error map and residual loss, we also plot the gradient correlation of corresponding models between nearby points, which is newly proposed to identify the propagation failure. A low gradient correlation value (the darker color) indicates that the area is hard to propagate. See Appendix \ref{['appdix:gradient_corr_vis']} for more results.
• Figure 2: Comparison between (a) FEMs and (b) PINNs. Blue arrows highlight quantities with direct interactions during training. Compared to FEMs, solution values of PINNs among different positions are under an implicit correlation by updating model parameter $\theta$ during training.
• Figure 3: Overall architecture of ProPINN from both (a) forward and (b) backward perspectives, where the single input point is augmented to point sets in multiscale regions, which can efficiently unite model parameter gradients on different positions within multiple regions.
• Figure 4: Visualization of model approximated solutions. Error map ($u_\theta-u$) is also plotted.
• Figure 5: Efficiency comparisons on Convection. Models under the single-point-processing paradigm are colored in blue, while models that consider point correlations are in red.

Theorems & Definitions (11)

• Theorem 3.1: Propagation in FEMs
• Remark 3.2: FEMs are under active propagation
• Remark 3.3: Physical meanings of Eq. \ref{['equ:fem_propagation']}
• Definition 3.4: Propagation failure in PINN
• Remark 3.5: Region propagation
• Theorem 3.6: Gradient correlation
• Theorem 3.8: Gradient correlation improvement
• Remark 3.9: Efficient design in ProPINN
• proof
• proof