An ILUES-based adaptive Gaussian process method for multimodal Bayesian inverse problems

TL;DR

The paper tackles multimodal Bayesian inverse problems with expensive forward models by introducing ILUES-augmented adaptive Gaussian process regression (ILUES-AGPR). It expresses the unnormalized posterior as $\tilde{\pi}(\theta|\boldsymbol{d}_{obs}) = \exp(f(\theta))\,p(\theta)$ and trains a GP surrogate for $f(\theta)$ using training data concentrated in high-probability regions via ILUES. An iterative scheme updates the auxiliary density and surrogate, and samples from the current posterior approximation with a Gaussian mixture MCMC that captures multiple modes. Numerical experiments on contamination-source problems demonstrate that ILUES-AGPR achieves accurate multimodal posteriors with far fewer forward-model evaluations than standard MCMC approaches, highlighting improved efficiency and robustness for complex Bayesian inverse problems.

Abstract

Inverse problems are prevalent in both scientific research and engineering applications. In the context of Bayesian inverse problems, sampling from the posterior distribution can be particularly challenging when the forward models are computationally expensive. This challenge is further compounded when the posterior distribution is multimodal. To address this issue, we propose a Gaussian process (GP)-based method to indirectly build surrogates for the forward model. Specifically, the unnormalized posterior density is expressed as a product of an auxiliary density and an exponential GP surrogate. Iteratively, the auxiliary density converges to the posterior distribution, starting from an arbitrary initial density. However, the efficiency of GP regression is highly influenced by the quality of the training data. Therefore, we utilize the iterative local updating ensemble smoother (ILUES) to generate high-quality samples that are concentrated in regions with high posterior probability. Subsequently, based on the surrogate model and mode information extracted using a clustering method, Markov chain Monte Carlo (MCMC) with a Gaussian mixed (GM) proposal is used to draw samples from the auxiliary density. Through numerical examples, we demonstrate that the proposed method can accurately and efficiently represent the posterior with a limited number of forward simulations.

Figures (6)

• Figure 1: Joint posterior distribution for Example 1 and its marginal probability density functions.
• Figure 2: Top row: Approximated posterior distributions at iterations $n=0$, $1$, and $2$. Blue dots in (a) represent ILUES ensemble points, while (b) and (c) show MCMC-generated samples. Bottom row: Ensemble samples at iterations $n=0$, $1$, and $2$. Dots in different colors represent distinct clusters, X marks indicate cluster centroids, and contours display the estimated PDF based on the ensemble points.
• Figure 3: (a) Posterior probability density functions estimated by DREAM and ILUES-AGPR. (b) Posterior probability density functions estimated by ILUES-AGPR and ILUES with two ensemble sizes, $N_e=50$ (ILUES-50) and $N_e=4000$ (ILUES-4000).
• Figure 4: One-dimensional (1D) and two-dimensional (2D) marginal posterior probability density functions obtained using DREAM-KZS for Example 2.
• Figure 5: Top row: ILUES step for the Ne80 case. (a) Prior samples (blue dots). (b) Ensemble samples after $n_0$ iterations with clustering, where different colors indicate different clusters. (c) Corresponding 1D and 2D marginal estimated probability density function (PDF), $\hat{p}_0$. Bottom row: ILUES step for the Ne150 case. (d) Prior samples (blue dots). (e) Ensemble samples after $n_0$ iterations with clustering, with colors representing clusters. (f) Corresponding 1D and 2D marginal estimated PDF, $\hat{p}_0$.