Uncertainty-Aware Diagnostics for Physics-Informed Machine Learning

TL;DR

This work addresses the challenge of multi-objective model selection in physics-informed machine learning by introducing the Physics-Informed Log Evidence (PILE) within a Gaussian-process-based Physics-Informed Kernel Learning (PIKL) framework. PILE unifies data fidelity, physics constraints, and regularization into a single uncertainty-aware metric derived from GP Bayes free energy, enabling robust hyperparameter tuning and kernel selection, including data-free kernel assessment. The authors demonstrate that PILE can guide bandwidth, regularization weights, and kernel choice, diagnose model misspecification, and, in a data-free limit, relate to Fredholm determinants that offer a principled kernel-criterion prior to data. The work further shows practical benefits through case studies on Poisson and convection PDEs, highlighting the potential of PILE to improve predictive accuracy and physics adherence in PIML, with broader applicability to nonlinear operators and extensions beyond GP formalisms.

Abstract

Physics-informed machine learning (PIML) integrates prior physical information, often in the form of differential equation constraints, into the process of fitting machine learning models to physical data. Popular PIML approaches, including neural operators, physics-informed neural networks, neural ordinary differential equations, and neural discrete equilibria, are typically fit to objectives that simultaneously include both data and physical constraints. However, the multi-objective nature of this approach creates ambiguity in the measurement of model quality. This is related to a poor understanding of epistemic uncertainty, and it can lead to surprising failure modes, even when existing statistical metrics suggest strong fits. Working within a Gaussian process regression framework, we introduce the Physics-Informed Log Evidence (PILE) score. Bypassing the ambiguities of test losses, the PILE score is a single, uncertainty-aware metric that provides a selection principle for hyperparameters of a PIML model. We show that PILE minimization yields excellent choices for a wide variety of model parameters, including kernel bandwidth, least squares regularization weights, and even kernel function selection. We also show that, even prior to data acquisition, a special 'data-free' case of the PILE score identifies a priori kernel choices that are 'well-adapted' to a given PDE. Beyond the kernel setting, we anticipate that the PILE score can be extended to PIML at large, and we outline approaches to do so.

Figures (5)

• Figure 1: Automatic hyperparameter selection with PILE. The PILE score and relative PPL2-G error sources for varying bandwidth, physics regularization, and data regularization parameters. Error bars show $\pm 2\widehat{\sigma}$ coverage, where $\widehat{\sigma}$ is the empirical standard deviation of PPL2-G (blue bars) and PILE (red bars). (Left) Bandwidth selection via minimizing the PILE score provides an accurate fit, balancing the data and physics generalization errors. (Middle, Right) After selecting the optimal bandwidth $h^*$, we sequentially minimize PILE first with respect to the physics regularization parameter $\rho$, and then with respect to the data regularization parameter $\gamma$. For small values of $\rho$ and $\gamma$, PILE diverges as the regression model overfits the noisy observations.
• Figure 2: Optimizing PILE prevents under- and over-smoothing. Qualitative plot of the negative effects of oversmoothing and undersmoothing in PIKL. Each panel shows $\widehat{f}$, $\widehat{g}$ on the left and right, respectively. When $h$ is too small, the derivative estimate is undersmoothed and irregular. When $h$ is too large, oversmoothing effects prevent the model from fitting the derivative. The optimal value of $h$ is identified by optimizing PILE.
• Figure 3: Data-free PILE landscape for anisotropic RBF kernel. Fredholm determinants of $k_{\theta, s}$ plotted for $\theta \in [-\pi, \pi]$ and $s \in [0.5, 1.5]$. This quantity is empirically minimized at $\theta^* \approx 1.41$, $s^* \approx 0.5$ (shown in red), but there are evidently symmetries in the landscape.
• Figure 4: Diagnostics with PILE: identifying model failure. Fitting the convection PDE, equation \ref{['eqn:convection-eqn']}, with an RBF kernel. There is no appropriate bandwidth for this problem; and the PILE score diagnosis this by selecting the "all zeros" solution.
• Figure 5: Diagnostics with PILE: hyperparameter selection after kernel adjustment. Fitting the convection PDE, equation \ref{['eqn:convection-eqn']}, with an anisotropic RBF kernel. By choosing the kernel with minimum Fredholm determinant, i.e., the "data-free" variant of the PILE score, we can automatically identify a "good" kernel for the continuity PDE.

Theorems & Definitions (9)

• Definition 4.1: Physics-Informed Log Evidence (PILE)
• Theorem 4.3
• Proposition A.1: Corollary 4.36 of steinwart2008support
• Proposition A.2
• Proposition A.3
• Theorem A.4: Representer Theorem
• Definition B.1
• Theorem B.2
• Definition B.3