A Black Box Variational Inference Scheme for Inverse Problems with Demanding Physics-Based Models

TL;DR

This paper addresses Bayesian inverse problems involving expensive, non-differentiable physics-based forward models. It develops ABRIS, an adaptive batch-sized regulated importance sampling BBVI, which reuses prior model calls via a mixture IS proposal and uses a moving-window sampling loop guided by ESS and gradient-correlation criteria to minimize new simulations. The approach achieves substantial reductions in forward-model evaluations and competitive posterior accuracy compared with MCMC and SMC across Gaussian density matching, a generalized Poisson calibration, and a solid-state-battery multiphysics calibration, demonstrating practicality for demanding physics-based problems. Ultimately, ABRIS provides a flexible, gradient-free framework that scales with computational budgets and can leverage parallelism, enabling efficient Bayesian calibration without model derivatives.

Abstract

Bayesian methods are particularly effective for addressing inverse problems due to their ability to manage uncertainties inherent in the inference process. However, employing these methods with costly forward models poses significant challenges, especially in the context of non-differentiable models, where the absence of likelihood model gradient information can result in high computational costs. To tackle this issue, we develop a novel Bayesian inference approach based on black box variational inference, utilizing importance sampling to reuse existing simulation model calls in the variational objective gradient estimation, without relying on forward model gradients. The novelty lies in a new batch-sequential sampling procedure, which only requires new model evaluations if the currently available model evaluations fail to yield a suitable approximation of the objective gradient. The resulting approach reduces computational costs by leading to variational parameter updates without requiring new model evaluations when possible, while adaptively increasing the number of model calls per iteration as needed. In combination with its black box nature, this new approach is suitable for inverse problems involving demanding physics-based models that lack model gradients. We demonstrate the efficiency gains of the proposed method compared to its baseline version, sequential Monte Carlo, and Markov-Chain Monte Carlo in diverse benchmarks, ranging from density matching to the Bayesian calibration of a nonlinear electro-chemo-mechanical model for solid-state batteries.

Figures (11)

• Figure 1: Visualization of the sampling of BBVI (left) and its IS-enhancement ABRIS (right). The x-axis is the index of the stochastic optimizer iteration. The y-axis describes the iterations associated with samples in $\Theta^i$ used in the IS approach. Points on $i=j$ indicate that the model is newly sampled at this iteration, and off-diagonal points indicate that it has been reused. The colour of the points correspond to importance sampling coefficient $\beta_j^i$ of the mixture component $j$ at iteration $i$.
• Figure 2: Gaussian match cases for $\dim{\boldsymbol{\theta}}\in \{1,2,4,8,16,32\}$. The number of model calls is plotted against window size $m$ for various samples per iteration $N$. The number of simulations is the average of 10 runs with different random seeds. BBVI indicates that no importance sampling was used.
• Figure 3: Number of simulations plotted over the optimiser iterations for $\dim{\boldsymbol{\theta}}=8$ with $N=2$ for various window sizes $m$. In the black zoom-in, the adaptive sampling strategy is visible. The red zoom-in shows the dominating periodic condition towards the end of the optimization.
• Figure 4: Ground truth $\zeta_\text{true}$ (left) and resulting model output (right).
• Figure 5: Comparison between posterior mean values and the ground truth $\boldsymbol{\zeta}_\text{true}$ in relative $L_2$-error and in $\boldsymbol{A}$-norm, weighting each elementwise constant coefficient with $A_{jj}=\zeta_\text{true, j}^2$, for the generalized Poisson problem.