Adaptive Gaussian Process Regression for Bayesian inverse problems

TL;DR

This work tackles Bayesian inverse problems with expensive forward models by coupling adaptive Gaussian Process Regression surrogates with a goal-oriented, sequential design strategy. The method interleaves posterior sampling via MCMC with training-set design by minimizing a posterior-focused error metric $E(\mathcal{D})$ under a computational budget, and by optimizing both training point locations and evaluation tolerances through a KL-divergence guided objective. Across 1D and 2D analytical tests, joint optimization of where to sample and how accurately to evaluate the forward model yields substantial reductions in the posterior approximation error compared to non-adaptive or tolerance-fixed approaches. The results demonstrate practical efficiency gains for surrogate-based Bayesian inversion in settings with costly forward models such as finite element simulations.

Abstract

We introduce a novel adaptive Gaussian Process Regression (GPR) methodology for efficient construction of surrogate models for Bayesian inverse problems with expensive forward model evaluations. An adaptive design strategy focuses on optimizing both the positioning and simulation accuracy of training data in order to reduce the computational cost of simulating training data without compromising the fidelity of the posterior distributions of parameters. The method interleaves a goal-oriented active learning algorithm selecting evaluation points and tolerances based on the expected impact on the Kullback-Leibler divergence of surrogated and true posterior with a Markov Chain Monte Carlo sampling of the posterior. The performance benefit of the adaptive approach is demonstrated for two simple test problems.

Figures (4)

• Figure 1: Impact of the likelihoods \ref{['eq:plug-in-likelihood']} and \ref{['eq:full-likelihood']} on the posterior for an illustrative inverse problem problem with forward model $y(p) = p^2 sin(p)$ and uniform prior on parameter space $[0,1]$. The horizontal lines show the actual measurement and the $2\sigma$ range of measurement noise. The marginal likelihood \ref{['eq:full-likelihood']} is wider due to including the GP variance, and avoids overconfident posteriors.
• Figure 2: Kullback-Leibler divergence of surrogated posterior and true posterior for different training designs over the computational work spent in the 1D example.
• Figure 3: Reduction of surrogate standard deviation of the $y_1$, i.e. $k=0$, component (left) and change of posterior distribution (right) between iterations 7 and 9. The computational work for each point is represented by its size. New points are added and some of the old points are refined. The true parameter used for creating the artificial measurements $y^m$ is denoted by a green star.
• Figure 4: Kullback-Leibler divergence of surrogated posterior and true posterior for different training designs over the computational work spent in the 2D example.