Model Error Embedding with Orthogonal Gaussian Processes

TL;DR

The paper addresses model inadequacy in complex physical simulations by embedding a model-error term within the computational model using a weight-space Gaussian process (GP) representation. To disentangle the model parameters from bias, it develops orthogonal GP (OGP) formulations, including linearized (LOGP) and regularized (ROGP) variants, and leverages likelihood-informed subspace (LIS) to manage the high dimensionality of the GP weights. The approach is demonstrated on linear, nonlinear, and advection–diffusion–reaction PDE problems, showing near-LS aligned parameter posteriors, decoupled GP weights, and accurate extrapolation back to prior predictive distributions. The combination of embedded GP error, orthogonality constraints, and LIS provides a principled, scalable framework for calibrated predictions with quantified model uncertainty in complex systems.

Abstract

Computational models of complex physical systems often rely on simplifying assumptions which inevitably introduce model error, with consequent predictive errors. Given data on model observables, the estimation of parameterized model-error representations, along with other model parameters, would be ideally done while separating the contributions of each of the two sets of parameters, in order to ensure meaningful stand-alone model predictions. This work builds an embedded model error framework using a weight-space representation of Gaussian processes (GPs) to flexibly capture model-error spatiotemporal correlations and enable inference with GP-embedding in non-linear models. To disambiguate model and model-error/bias parameters, we extend an existing orthogonal GP method to the embedded model-error setting and derive appropriate orthogonality constraints. To address the increased dimensionality introduced by the GP representation, we employ the likelihood-informed subspace method. The construction is demonstrated on linear and non-linear examples, where it effectively corrects model predictions to match data trends. Extrapolation beyond the training data recovers the prior predictive distribution, and the orthogonality constraints lead to meaningful stand-alone model predictions and nearly uncorrelated posteriors between model and model-error parameters.

Figures (26)

• Figure 1: Panels (a) and (b) show the comparison of plus/minus one standard deviation obtained using the FS view (dashed black) and the WS view with $m = 10$ (red), $20$ (green), and $40$ (blue) for correlation lengths 0.2 and 1 respectively.
• Figure 2: Comparison of PFP mean and plus/minus three standard deviations using FS and WS formulations. GP covariance function hyperparameters are $\sigma=1$ and $l=1$. In panels (a) and (b) the noise variance is set to $10^{-10}$ and in panels (c) and (d) it is set to 0.1.
• Figure 3: Frames (a) and (b) show marginal priors, posteriors and least-squares estimates for $\lambda_0$ and $\lambda_1$ obtained with KOH. Panel (c) shows the truth function, data points and push forward posterior predictions. PFP lines show the mean prediction and bands show $\pm$ three standard deviations.
• Figure 4: Frames (a) and (b) show marginal priors, posteriors and least-squares estimates for $\lambda_0$ and $\lambda_1$ obtained with OGP. Panel (c) shows the truth function, data points and push forward posterior predictions. PFP lines show the mean prediction and bands show $\pm$ three standard deviations.
• Figure 5: Frames (a) and (b) show the joint posterior distributions of model parameters and GP weights for $N=20$ and $m=20$ obtained with KOH and OGP respectively.