Robust Experimental Design via Generalised Bayesian Inference

TL;DR

We address the fragility of Bayesian optimal experimental design under model misspecification and outliers by proposing Generalised Bayesian Optimal Experimental Design (GBOED). GBOED extends Bayesian design to generalized Bayesian inference using Gibbs posteriors and introduces Gibbs Expected Information Gain (Gibbs EIG) as a tractable, information-theoretic design criterion, computable via importance sampling and nested Monte Carlo when paired with scoring-rule losses. The framework leverages power-like losses, score matching, and inverse-MQ kernel weighting to achieve robustness to misspecification, with an exponential-decay strategy for IMQ parameters that balances robustness and learning. Across linear regression, pharmacokinetics, and location finding tasks, GBOED demonstrates improved predictive performance and more robust exploration under asymmetric outliers and incorrect noise assumptions, especially in higher dimensions. These results suggest GBOED provides a practical pathway to reliable sequential experimentation when domain models are imperfect or uncertain.

Abstract

Bayesian optimal experimental design is a principled framework for conducting experiments that leverages Bayesian inference to quantify how much information one can expect to gain from selecting a certain design. However, accurate Bayesian inference relies on the assumption that one's statistical model of the data-generating process is correctly specified. If this assumption is violated, Bayesian methods can lead to poor inference and estimates of information gain. Generalised Bayesian (or Gibbs) inference is a more robust probabilistic inference framework that replaces the likelihood in the Bayesian update by a suitable loss function. In this work, we present Generalised Bayesian Optimal Experimental Design (GBOED), an extension of Gibbs inference to the experimental design setting which achieves robustness in both design and inference. Using an extended information-theoretic framework, we derive a new acquisition function, the Gibbs expected information gain (Gibbs EIG). Our empirical results demonstrate that GBOED enhances robustness to outliers and incorrect assumptions about the outcome noise distribution.

Figures (12)

• Figure 1: Designs selected by both BOED and GBOED in a 2D location finding example in well-specified and misspecified scenarios. Designs that cluster around the objects (red crosses) are most informative in determining the objects' locations. Top left: In the well-specified setting, BOED selects designs that cluster around the objects. Top right: When the model is misspecified, BOED clusters around irrelevant regions with no objects. Bottom row: GBOED effectively avoids clustering in irrelevant regions.
• Figure 2: Methods compared under the asymmetric outlier scenario on three experimental design problems (columns); mean MMD (line) during experimentation and standard error (SE) shaded. Insets are zoomed in versions of the plots. Top row displays the outlier scenario using different loss functions. Bottom row displays the outlier scenario for GBOED but with alternative acquisition functions.
• Figure 3: Comparison of different downweighting rates $c$ on the Gibbs EIG under different priors, for the Bayesian linear regression problem. Left: poor prior (unit Gaussian, larger posterior variance). Right: good prior (close to the true posterior, smaller posterior variance). Smaller $c$ values tend to cause queries slightly away from the extremes, with more noise compared to the other curves. Large $c$ values have EIG curves close to the unweighted score matching loss.
• Figure 4: Comparison of different learning rates for computing the Gibbs EIG using the negative log-likelihood loss, under the Bayesian linear regression problem. Marked crosses are the 10 designs with the greatest (Gibbs) EIG for a particular curve. Bottom: The Gibbs EIG at $\omega = 0.1$. EIG values are lower than the other two curves, and designs slightly further away from the extremes have greater EIG. Middle: The Gibbs EIG at $\omega = 0.5$. Less smooth EIG curve than the curve above, with some indication of designs with greater EIG values being somewhat away from the extremes. Top: The BEIG ($\omega = 1$). Greatest EIG values are at the most extreme ends of the design space, and the EIG curve is much smoother than the other two.
• Figure 5: Comparison of different values of the rate $b$ in the exponential decay method for selecting $c$ in $r_{\mathrm{IMQ}}$. $b = 0.04$ results in the slowest decrease of $c$ per experiment, whereas $b = 0.12$ results in the fastest decrease.

Theorems & Definitions (15)

• Definition 1: KL divergence
• Definition 2: EIG
• Definition 3: Pseudo-rv
• Definition 4: Pseudo-expectation $\widetilde{\mathbb{E}}$
• Definition 5: Pseudo-KL divergence
• Definition 6: Pseudo-joint density
• Definition 7: Pseudo-mutual information
• Definition 8: Gibbs EIG
• Theorem 1
• Definition 9: Gibbs EIG NMC estimator