Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sequential Bayesian Experimental Design for Prediction in Physical Experiments Informed by Computer Models

Hao Zhu, Markus Hainy

Abstract

In many scientific and engineering domains, physical experiments are often costly, non-replicable, or time-consuming. The Kennedy and O'Hagan (KOH) model framework has become a widely used approach for combining simulator runs with limited experimental observations. Under a Bayesian implementation, the simulator output, model discrepancy, and observation noise are jointly modeled by coupled Gaussian processes, followed by coherent posterior inference and uncertainty quantification. This work presents a genuinely sequential Bayesian experimental design (BED) framework explicitly aimed at improving the predictive performance of the KOH model. We employ a mutual information (MI)-based criterion and develop a hybrid variant that integrates it with measures of local model complexity, leading to significantly more efficient design decisions. We further show theoretically that the MI-based criterion is more comprehensive and robust than the classical integrated mean squared prediction error (IMSPE) minimization criterion, especially when the model is highly uncertain in the early stages of the experiment. To mitigate the computational burden of fully Bayesian inference and the ensuing BED process, we propose two acceleration strategies - Gaussian Mixture Compression and Schur complement and rank-one update - which together substantially reduce runtime. Finally, we demonstrate the effectiveness of the proposed methods through both a synthetic example and a real biochemical case study, and compare them against several classical design criteria under sequential (offline) and adaptive (online) BED settings.

Sequential Bayesian Experimental Design for Prediction in Physical Experiments Informed by Computer Models

Abstract

In many scientific and engineering domains, physical experiments are often costly, non-replicable, or time-consuming. The Kennedy and O'Hagan (KOH) model framework has become a widely used approach for combining simulator runs with limited experimental observations. Under a Bayesian implementation, the simulator output, model discrepancy, and observation noise are jointly modeled by coupled Gaussian processes, followed by coherent posterior inference and uncertainty quantification. This work presents a genuinely sequential Bayesian experimental design (BED) framework explicitly aimed at improving the predictive performance of the KOH model. We employ a mutual information (MI)-based criterion and develop a hybrid variant that integrates it with measures of local model complexity, leading to significantly more efficient design decisions. We further show theoretically that the MI-based criterion is more comprehensive and robust than the classical integrated mean squared prediction error (IMSPE) minimization criterion, especially when the model is highly uncertain in the early stages of the experiment. To mitigate the computational burden of fully Bayesian inference and the ensuing BED process, we propose two acceleration strategies - Gaussian Mixture Compression and Schur complement and rank-one update - which together substantially reduce runtime. Finally, we demonstrate the effectiveness of the proposed methods through both a synthetic example and a real biochemical case study, and compare them against several classical design criteria under sequential (offline) and adaptive (online) BED settings.
Paper Structure (24 sections, 8 theorems, 118 equations, 8 figures, 2 tables, 2 algorithms)

This paper contains 24 sections, 8 theorems, 118 equations, 8 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Review of Kennedy & O'Hagan Model
  4. Design Objectives and Strategies
  5. Classical Design Criteria
  6. Mutual Information–Based Design Criterion
  7. Mutual Information Gain for Improving Future Predictions
  8. Hybrid Criterion with Local Complexity Adjustment
  9. Theoretical Properties
  10. Computation Strategies
  11. Gaussian Mixture Compression
  12. Precomputation and Schur Complement
  13. Overall Computational Complexity
  14. Experiments
  15. Evaluation Criteria
  16. ...and 9 more sections

Key Result

Theorem 1

Assume that, (i) conditional on $\boldsymbol{\Omega}$, $(\mathbf y^*,\mathbf y^{\mathrm{new}}_{(b)})$ is jointly Gaussian, (ii) $\boldsymbol{\Sigma}^*_{(b-1)}$ is symmetric positive definite with eigenvalues $0<\lambda_{\min}\le\lambda_{\max}<\infty$. Define the covariance reduction and IMSPE reduct As the sequential design rounds proceed, the following small-update regime holds: Then there exist

Figures (8)

  • Figure 1: Evolution of the joint predictive distribution at two locations under a 5-run sequential (offline) design of experiments (SDE) using the proposed MI-based criterion.
  • Figure 2: Simulation case: comparison of (a) the original model regression and (b) after design model regression.
  • Figure 3: Simulation case: comparison of average (a) MSE and (b) CRPS for each criterion across 10 design rounds.
  • Figure 4: Simulation case: comparison of final-round (a) MSE and (b) CRPS for each criterion.
  • Figure 5: Simulation case: comparison of (a) $\Delta \text{MSE}$ and (b) $\Delta\text{CRPS}$ for the MI-based criterion and IMSPE minimization over 20 rounds.
  • ...and 3 more figures

Theorems & Definitions (16)

  • Theorem 1: Local asymptotic relationship between MI and IMSPE
  • proof
  • Lemma 1: Smoothness and bounded derivatives of the outer-layer function
  • proof
  • Corollary 1: Convergence of the NMC estimator
  • proof
  • Corollary 2: Bias and almost sure convergence of the NMC estimator
  • proof
  • Theorem 1: Local asymptotic relationship between MI and IMSPE
  • proof
  • ...and 6 more