E-PINNs: Epistemic Physics-Informed Neural Networks

TL;DR

E-PINNs is proposed, a framework that uses a small network, the epinet, to efficiently quantify epistemic uncertainty in PINNs and works as an add-on to existing, pre-trained PINNs with a small computational overhead.

Abstract

Physics-informed neural networks (PINNs) have demonstrated promise as a framework for solving forward and inverse problems involving partial differential equations. Despite recent progress in the field, it remains challenging to quantify uncertainty in these networks. While techniques such as Bayesian PINNs (B-PINNs) provide a principled approach to capturing epistemic uncertainty through Bayesian inference, they can be computationally expensive for large-scale applications. In this work, we propose Epistemic Physics-Informed Neural Networks (E-PINNs), a framework that uses a small network, the epinet, to efficiently quantify epistemic uncertainty in PINNs. The proposed approach works as an add-on to existing, pre-trained PINNs with a small computational overhead. We demonstrate the applicability of the proposed framework in various test cases and compare the results with B-PINNs using Hamiltonian Monte Carlo (HMC) posterior estimation and dropout-equipped PINNs (Dropout-PINNs). In our experiments, E-PINNs achieve calibrated coverage with competitive sharpness at substantially lower cost. We demonstrate that when B-PINNs produce narrower bands, they under-cover in our tests. E-PINNs also show better calibration than Dropout-PINNs in these examples, indicating a favorable accuracy-efficiency trade-off.

Figures (23)

• Figure 1: Schematic diagram of the E-PINN architecture. The base PINN (red) produces a deterministic prediction $u_\xi$. The epinet (green) receives as input the augmented feature vector $\tilde{x} = \texttt{sg}([\,x,\; h_{\text{penultimate}}(x;\xi)\,])$, where sg blocks gradients to the base network, together with a random epistemic index $z \sim P_z$. Internally, the epinet comprises two neural networks: a trainable network producing $e_\eta^L$ and a non-trainable prior network with fixed parameters producing $e^P$. The final epinet output is $e_\eta = e_\eta^L + \alpha\, e^P$, a weighted sum controlled by the hyperparameter $\alpha$. The sum of the epinet output and the base PINN output yields the stochastic E-PINN prediction $u_\theta$.
• Figure 2: Mean and uncertainty predictions for $u$ in the 1D Poisson equation (physics-only) using (a) E-PINN (top row) and (b,c) Dropout-PINN (bottom row; 5% and 10%).
• Figure 3: Mean and uncertainty predictions for $u$ in the 1D Poisson equation (physics + interior data, Gaussian noise with $\sigma = 0.10\,\lVert u\rVert_{\infty}$) using (a) E-PINN (b) Dropout-PINN (5%) (c) Dropout-PINN (10%) (d) B-PINN.
• Figure 4: Mean and uncertainty predictions for $u$ in 1D nonlinear Poisson (physics-only, no interior data) using (a) E-PINN (b) Dropout-PINN (5%) (c) Dropout-PINN (10%). B-PINN is not evaluated in physics-only.
• Figure 5: Mean and uncertainty predictions for $u$ in 1D nonlinear Poisson (Gaussian noise, $\sigma = 0.30\,\lVert u\rVert_{\infty}$) using (a) E-PINN (b) Dropout-PINN (5%) (c) Dropout-PINN (10%) (d) B-PINN.