A Hybrid Two-level MCMC Framework to Accelerate Posterior Mean Estimation with Deep Learning Surrogates for Bayesian Inverse Problems
Juntao Yang, Jeff Adie, Simon See, Adriano Gualandi, Gianmarco Mengaldo
TL;DR
The paper tackles the computational bottleneck of Bayesian inverse problems governed by PDEs, where repeated forward solves are costly. It introduces a hybrid two-level MCMC method that couples a fast DL surrogate-based base chain with a short high-fidelity correction chain, leveraging a telescoping-sum estimator to recover the posterior mean of quantities of interest with the same $\mathcal{O}(h)$ accuracy as full numerical MCMC but at a fraction of the cost. The authors provide a rigorous error decomposition showing a priori $\mathcal{O}(2^{-L})$ convergence, and validate the theory across Poisson, nonlinear reaction-diffusion, and Navier–Stokes problems under both uniform and Gaussian priors, reporting substantial speedups (often an order of magnitude or more) while preserving accuracy. This framework offers a principled way to exploit DL surrogates in Bayesian inference for PDEs, with potential extensions to ensemble Kalman filters and sequential Monte Carlo methods, and establishes a pathway to rigorousDL-based posterior error control in practical applications.
Abstract
Bayesian inverse problems arise in various scientific and engineering domains, and solving them can be computationally demanding. This is especially the case for problems governed by partial differential equations, where the repeated evaluation of the forward operator is extremely expensive. Recent advances in Deep Learning (DL)-based surrogate models have shown promising potential to accelerate the solution of such problems. However, despite their ability to learn from complex data, DL-based surrogate models generally cannot match the accuracy of high-fidelity numerical models, which limits their practical applicability. We propose a novel hybrid two-level Markov Chain Monte Carlo (MCMC) method that combines the strengths of DL-based surrogate models and high-fidelity numerical solvers to {compute the posterior mean of Quantities of Interest (QoI) in} Bayesian inverse problems governed by partial differential equations. The intuition is to leverage the evaluation speed of a DL-based surrogate model as the base chain, and correct its errors using a limited number of high-fidelity numerical model evaluations in a correction chain; hence its name hybrid two-level MCMC method. Through a detailed theoretical analysis, we show that our approach can achieve the same accuracy as a pure numerical MCMC method while requiring only a small fraction of the computational cost. The theoretical analysis is further supported by several numerical experiments, namely a Poisson, a non-linear reaction-diffusion, and a Navier-Stokes equation. The proposed hybrid framework can be generalized to other approaches such as the ensemble Kalman filter and sequential Monte Carlo methods.