Bayesian Surrogate Training on Multiple Data Sources: A Hybrid Modeling Strategy

TL;DR

This work introduces a Bayesian surrogate framework to fuse simulation data and real-world measurements for training surrogates, addressing structural misspecification in simulators. It presents two core approaches: posterior predictive weighting (PW) of separately trained surrogates and power-scaling (PS) of likelihoods to train a single surrogate, both governed by a weight parameter $\beta$. Through synthetic and real-data case studies (including SIR and COVID-19 data), the paper demonstrates that hybrid training can improve predictive accuracy and uncertainty calibration, while offering diagnostic insights into model misspecifications. The framework is general, potentially extensible to multiple data sources and inputs, and provides guidance on choosing the weighting strategy to balance predictive performance and interpretability in practice.

Abstract

Surrogate models are often used as computationally efficient approximations to complex simulation models, enabling tasks such as solving inverse problems, sensitivity analysis, and probabilistic forward predictions, which would otherwise be computationally infeasible. During training, surrogate parameters are fitted such that the surrogate reproduces the simulation model's outputs as closely as possible. However, the simulation model itself is merely a simplification of the real-world system, often missing relevant processes or suffering from misspecifications e.g., in inputs or boundary conditions. Hints about these might be captured in real-world measurement data, and yet, we typically ignore those hints during surrogate building. In this paper, we propose two novel probabilistic approaches to integrate simulation data and real-world measurement data during surrogate training. The first method trains separate surrogate models for each data source and combines their predictive distributions, while the second incorporates both data sources by training a single surrogate. We show the conceptual differences and benefits of the two approaches through both synthetic and real-world case studies. The results demonstrate the potential of these methods to improve predictive accuracy, predictive coverage, and to diagnose problems in the underlying simulation model. These insights can improve system understanding and future model development.

Figures (15)

• Figure 1: Schematic overview of the four discussed surrogate approaches (four columns from left to right). In the top row, we illustrate the training process of each surrogate model, indicating the data sources used: either simulation data $\mathcal{D}_S$, real data $\mathcal{D}_R$, or both. The posterior (solid colored line) distributions of the surrogate model coefficients are displayed, where the posterior depicts the outcome of the training process based on the used data. In the bottom row, we show the posterior predictions of the surrogate model for the output variable $\hat{y}_R$. We indicate for each approach whether an additional inference step for unknown parameters using real data $\mathcal{D}_R$ is needed. Finally, the colored arrow depicts the role of the weighting factor $\beta$.
• Figure 2: Case study 1: Illustration of the simulation model and the real-world data. The black line depicts the predictive mean of the simulation model based on the prior parameter distribution $p(\omega_R)$, and the gray interval depicts the 90$\%$ credible interval of the predictive prior. The blue line depicts the synthetic truth. The dark blue data represents the (noisy) real training data, while the pink is the real OOS test data, and the light blue is the real OOD test data, respectively.
• Figure 3: Case study 1: Predictive mean posteriors as obtained from the hybrid surrogates with varied weighting factor $\beta$. Top row: results using the power-scaling approach, bottom row: results from the posterior predictive weighting approach. Each line represents one of 200 posterior samples from the predictive mean posterior distribution, color-coded by the corresponding $\omega_R$ value. The dark and light blue points show train data and out-of-distribution test data, respectively, as also shown in the data setup in Fig. \ref{['fig:case_study_1_setup']}. To enhance clarity, out-of-sample test data is omitted.
• Figure 4: Case study 1: Posterior predictives as obtained from the hybrid surrogates with varied weighting factor $\beta$. Top row: results using the power-scaling approach, bottom row: results from the posterior predictive weighting approach. We display the 50%, 90%, 99% credible intervals along with the median prediction. The dark and light blue points show train data and out-of-distribution test data, respectively, as also shown in the data setup in Fig. \ref{['fig:case_study_1_setup']}. To enhance clarity, out-of-sample test data is omitted.
• Figure 5: Case study 1: ELPD and RMSE of power-scaling and posterior predictive weighting on test data. The test data splits are out-of-distribution (OOD), out-of-sample in-distribution (OOS), and their combination (OOS/OOD) using 1/1 and 5/1 sample size ratios. For ELPD, higher is better; for RMSE, lower is better.