Active Learning with Adaptive Non-Stationary Kernel for Continuous-Fidelity Surrogate Models

TL;DR

This work tackles the computational burden of high-fidelity simulations by embedding a continuous fidelity parameter into a Gaussian process surrogate and introducing an adaptive non-stationary kernel that flexibly captures fidelity-related behavior. It combines this with an IMSPE-based, cost-aware active learning strategy that sequentially selects design points to minimize global predictive uncertainty of the true solution while accounting for simulation costs. The core contributions are a novel Lifted Brownian–style kernel $K_{\gamma}$ for fidelity interactions, a practical RMLE-based inference workflow yielding BLUP predictions, and a cost-aware, gradient-enabled IMSPE criterion enabling efficient sequential design. Case studies in Poisson, turbine blade stress, and wave equations demonstrate substantial predictive gains from the multi-fidelity, adaptive framework, with an accompanying R package provided for practitioners.

Abstract

Simulating complex physical processes across a domain of input parameters can be very computationally expensive. Multi-fidelity surrogate modeling can resolve this issue by integrating cheaper simulations with the expensive ones in order to obtain better predictions at a reasonable cost. We are specifically interested in computer experiments where real-valued fidelity parameters determine the fidelity of the numerical output, such as finite element methods. In these cases, integrating this fidelity parameter in the analysis enables us to make inference on fidelity levels that have not been observed yet. Such models have been developed, and we propose a new adaptive non-stationary kernel function which more accurately reflects the behavior of computer simulation outputs. In addition, we develop an active learning strategy based on the integrated mean squared prediction error (IMSPE) to identify the best design points across input parameters and fidelity parameters, while taking into account the computational cost associated with the fidelity parameters. We illustrate this methodology through numerical examples and applications to finite element methods. An $\textsf{R}$ package for the proposed methodology is provided in an open repository.

Figures (8)

• Figure 1: Results of FEM simulations at the one input point for a sequence of mesh sizes on the Poisson (middle and right) and jet blade (left panel) case studies; the 3 columns correspond to the 3 different case study problems at hand. The top row shows the response of interests $y$ depending on the mesh size with the horizontal dot line showing the convergence value and the vertical dot line at mesh size $t = 0$, while the bottom row shows the increments depending on the mesh size.
• Figure 2: Top: Prediction of our model with the true function (black line), the prediction mean (gray line), and the 95% prediction band (shaded region). The color of the design points indicate the corresponding fidelity parameter, with a darker color indicating a lower value. Bottom: Active learning criterion surface (bottom). In both panels, the bullet points $(\bullet)$ represent the current design locations, while the crosses $(\times)$ indicate the best next design point according to the criterion.
• Figure 3: RMSE (in logarithmic scale) for the numerical study for each method, and all combinations of $\phi_2^2$ and $\gamma$.
• Figure 4: RMSE (in logarithmic scale) for the Poisson's equation case study with respect to the simulation cost. Solid lines indicate the average over 10 repetitions, while shaded regions represent the range. The response of interest is the average (left) or the maximum (right) over the domain $D$.
• Figure 5: Boxplots of the final RMSE in logarithmic scale for the Poisson's equation case study across 10 repetitions. The response of interest is the average (left) or the maximum (right) over the domain $D$.

Theorems & Definitions (2)

• Theorem 1
• proof