Bayesian Model Inference using Bayesian Quadrature: the Art of Acquisition Functions and Beyond

TL;DR

The paper tackles Bayesian model inference under complex posteriors that are multimodal, nonlinear, and sharply peaked. It advances Bayesian Quadrature by reinterpreting acquisition strategies, deriving semi-analytical quadrature rules for model evidence, and introducing four acquisition functions (PUQ, PVC, PLUR, PEUR), with further extensions to Transitional Bayesian Quadrature (TBQ) to handle highly divergent posteriors via tempering. The authors provide closed-form or MC-estimated formulations, establish stopping criteria and sampling schemes, and demonstrate substantial reductions in likelihood evaluations across 2D to 10D problems and a battery cooling application, while achieving accurate posterior and evidence estimates. The work highlights that no single method dominates across all problems but shows that the proposed acquisitions, especially PEUR, offer robust, efficient performance for low- to mid-dimensional Bayesian inverse problems with uncertain, multi-modal posteriors and sharp evidence surfaces, enabling practical uncertainty quantification with limited model evaluations.

Abstract

Estimating posteriors and the associated model evidences is a core issue of Bayesian model inference, and can be of great challenge given complex features of the posteriors such as multi-modalities of unequal importance, nonlinear dependencies and high sharpness. Bayesian Quadrature (BQ) has emerged as a competitive framework for tackling this challenge, as it provides flexible balance between computational cost and accuracy. The performance of a BQ scheme is fundamentally dictated by the acquisition function as it exclusively governs the generation of integration points. After reexamining one of the most advanced acquisition function from a prospective inference perspective and reformulating the quadrature rules for prediction, four new acquisition functions, inspired by distinct intuitions on expected rewards, are primarily developed, all of which are accompanied by elegant interpretations and highly efficient numerical estimators. Mathematically, these four acquisition functions measure, respectively, the prediction uncertainty of posterior, the contribution to prediction uncertainty of evidence, as well as the expected reduction of prediction uncertainties concerning posterior and evidence, and thus provide flexibility for highly effective design of integration points. These acquisition functions are further extended to the transitional BQ scheme, along with several specific refinements, to tackle the above-mentioned challenges with high efficiency and robustness. Effectiveness of the developments is ultimately demonstrated with extensive benchmark studies and application to an engineering example.

Figures (15)

• Figure 1: Reference results of unnormalized posteriors and the associated model evidences of the four 2D examples .
• Figure 2: Results of the posteriors defined by $\mathcal{U}_1$, which are estimated by BQ algorithm driven by the four acquisition functions, together with the training points and the mean estimate of model evidence.
• Figure 3: Results of the posteriors defined by $\mathcal{U}_2$, generated with the BQ algorithm.
• Figure 4: Results of the posteriors defined by $\mathcal{U}_3$, generated with the BQ algorithm.
• Figure 5: Results of the posteriors of the forth 2D example, generated with the BQ algorithm.