Multi-fidelity Gaussian process surrogate modeling for regression problems in physics

TL;DR

This work tackles data scarcity in surrogate modeling of expensive physics simulations by systematically evaluating multi-fidelity Gaussian process surrogates. It compares linear AR1, non-linear autoregressive (NARGP, GPDF, GPDFC), and deep Gaussian process (DGP) approaches, and introduces delay fusion and composite kernels (GPDF/GPDFC) to extend fidelity handling beyond two levels. Across academic benchmarks and real-world terramechanics and plasma microturbulence problems, multi-fidelity surrogates generally beat single-fidelity GP at equivalent cost, with performance hinging on the fidelity relationship and problem structure; structured kernels substantially improve DGP-based models. The results provide practical guidance on method selection, calibration needs, and the potential of MFGP approaches to accelerate physics-based simulations while maintaining predictive reliability.

Abstract

One of the main challenges in surrogate modeling is the limited availability of data due to resource constraints associated with computationally expensive simulations. Multi-fidelity methods provide a solution by chaining models in a hierarchy with increasing fidelity, associated with lower error, but increasing cost. In this paper, we compare different multi-fidelity methods employed in constructing Gaussian process surrogates for regression. Non-linear autoregressive methods in the existing literature are primarily confined to two-fidelity models, and we extend these methods to handle more than two levels of fidelity. Additionally, we propose enhancements for an existing method incorporating delay terms by introducing a structured kernel. We demonstrate the performance of these methods across various academic and real-world scenarios. Our findings reveal that multi-fidelity methods generally have a smaller prediction error for the same computational cost as compared to the single-fidelity method, although their effectiveness varies across different scenarios.

Figures (16)

• Figure 1: A kernel function defines the family of the functions with which we are approximating the target function. In this example, we use squared-exponential or RBF kernel and visualize a) a prior distribution over the functions. b) In the posterior distribution in the second plot, newly observed training data points, depicted here in black, narrow the space of potential functions from the initial family, defined by the RBF kernel.
• Figure 2: It is difficult to approximate the function $f_{h}(x)$ (orange) using only $x$ as an input, because it is highly oscillatory. Including the function $f_{l}(x)$ (blue) as an additional input means only the green, smooth manifold must be approximated, which can be done accurately with only a few data points. The example is adapted from perdikaris2017nonlinear.
• Figure 3: Flowchart showing the training and prediction steps of NARGP surrogate for a three-fidelity scenario. $f_1$, $f_2$, and $f_3$ represent the function at each fidelity. $g_1$, $g_2$, and $g_3$ represent the surrogate of the corresponding fidelity. Training the NARGP model for a particular level involves evaluating the model of that fidelity level and the previous fidelity level. We draw samples to make predictions for a particular level, which involves drawing samples from the surrogate of the previous level and feeding them as input to the surrogate of that level. We can then use the samples to obtain the required statistical moments of the predictions by marginalizing the samples.
• Figure 4: Flowchart showing the training and prediction steps of Deep Gaussian process surrogate for three fidelity scenarios. $f_1$, $f_2$, and $f_3$ represent the function at each fidelity. $g_1$, $g_2$, and $g_3$ represent the surrogate of the corresponding fidelity. Training the DGP model for a particular level involves evaluating the model of that fidelity level and the surrogate of the previous fidelity level, which further involves evaluating the one-level lower surrogate. This creates a cascading effect until we reach the lowest fidelity surrogate. The prediction process is similar to the full GP, which involves drawing and marginalizing samples.
• Figure 5: Visualization of different academic problems. The low and high-fidelity functions are visualized in each sub-plot by orange and blue curves, respectively.