Improving PINNs By Algebraic Inclusion of Boundary and Initial Conditions

TL;DR

This work tackles the instability of Physics-Informed Neural Networks (PINNs) for PDEs by algebraically embedding boundary and initial conditions into the predictor, reducing reliance on loss-based penalties. Through fully unsupervised experiments on Poisson, Burgers, and KdV equations across multiple dimensions, the authors show that boundary-inclusive models excel for Poisson and 2D Burgers, while initial-inclusive models excel for 3D Burgers and KdV solitons, revealing dimension-dependent preferences. The study benchmarks against Deep Ritz Method (DRM) and XPINNs, reporting order-of-magnitude improvements in accuracy without altering the training algorithm, and provides detailed empirical comparisons across six loss configurations. The results advocate a principled choice of conditioning—boundary vs initial—based on the PDE and dimensionality, and they highlight opportunities for theoretical analysis of convergence and extension to other elliptic problems. The work has practical impact for robust unsupervised PDE solvers and informs design choices in physics-informed learner architectures.

Abstract

"AI for Science" aims to solve fundamental scientific problems using AI techniques. As most physical phenomena can be described as Partial Differential Equations (PDEs) , approximating their solutions using neural networks has evolved as a central component of scientific-ML. Physics-Informed Neural Networks (PINNs) is the general method that has evolved for this task but its training is well-known to be very unstable. In this work we explore the possibility of changing the model being trained from being just a neural network to being a non-linear transformation of it - one that algebraically includes the boundary/initial conditions. This reduces the number of terms in the loss function than the standard PINN losses. We demonstrate that our modification leads to significant performance gains across a range of benchmark tasks, in various dimensions and without having to tweak the training algorithm. Our conclusions are based on conducting hundreds of experiments, in the fully unsupervised setting, over multiple linear and non-linear PDEs set to exactly solvable scenarios, which lends to a concrete measurement of our performance gains in terms of order(s) of magnitude lower fractional errors being achieved, than by standard PINNs. The code accompanying this manuscript is publicly available at, https://github.com/MorganREN/Improving-PINNs-By-Algebraic-Inclusion-of-Boundary-and-Initial-Conditions

Figures (25)

• Figure 1: Fractional Error of Different Loss Functions with the Best Performance on the Right ($d= 10$). Left : $\mathcal{\hat{{\mathcal{R}}}}_{\rm energy, loss-with-boundary-penalty}$ (\ref{['compare_loss']}) i.e the "Deep Ritz Method". Right: $\mathcal{\hat{{\mathcal{R}}}}_{\rm residual, boundary-included, \lambda = 5}$ (\ref{['best_empirical_risk']}).
• Figure 2: Heatmap Representations for the True Solution and the Neural Approximant (Obtained By Minimizing the Boundary-Included Model) of the $2-$Dimensional Burgers' PDE Solved on Different Time Intervals Getting Close to the Blow-up Time Instant.
• Figure 3: Change of Empirical and Population Risk with Epochs of Training for Solving for the Target 3D Burgers' PDE at $\nu=0.01$ (Left Column) and $\nu=1$ (Right Column). The Three Lines in Each Plot Correspond to The Three Models Tried for Each $\nu$. Note That, in the Figures Above, the Legend of "vanilla model" Refers to the Standard PINN Method.
• Figure 4: In the Figures Above We See a Comparison of the Outputs Between The Different Models Trying to Solve for the 3-Soliton Solution of the KdV PDE. Note That, in the Figures Above, the Legend of "vanilla model" Refers to the Standard PINN Method.
• Figure 5: Empirical Loss of Different Loss Function ($d = 10$). Left : $\mathcal{\hat{{\mathcal{R}}}}_{\rm energy, boundary-included, \lambda = 1}$ (\ref{['Energyloss-outscale']}). Right: $\mathcal{\hat{{\mathcal{R}}}}_{\rm residual, boundary-included, \lambda = 1}$ (\ref{['Residualloss-outscale']}).