Taylor Approximation Variance Reduction for Approximation Errors in PDE-constrained Bayesian Inverse Problems

TL;DR

The paper tackles the computational burden of accounting for surrogate-model errors in PDE-constrained Bayesian inverse problems by introducing a Taylor expansion-based control variate framework within the Bayesian approximation error (BAE) approach. By representing the forward map’s first- and second-order derivatives as control variates, it achieves significant variance reduction in the estimation of the approximation-error mean and covariance, enabling faster convergence with far fewer offline samples. The method is derivative-based, mesh-agnostic, and scales with data-intrinsic dimension, and it is complemented by a sample-free deterministic option. Demonstrations on two high-dimensional PDE inverse problems (a linear Robin boundary problem and a nonlinear diffusion/PAT-inspired problem) show substantial improvements in posterior accuracy and uncertainty quantification, along with practical extensions like QMC and multifidelity strategies.

Abstract

In numerous applications, surrogate models are used as a replacement for accurate parameter-to-observable mappings when solving large-scale inverse problems governed by partial differential equations (PDEs). The surrogate model may be a computationally cheaper alternative to the accurate parameter-to-observable mappings and/or may ignore additional unknowns or sources of uncertainty. The Bayesian approximation error (BAE) approach provides a means to account for the induced uncertainties and approximation errors (between the accurate parameter-to-observable mapping and the surrogate). The statistics of these errors are in general unknown a priori, and are thus calculated using Monte Carlo sampling. Although the sampling is typically carried out offline the process can still represent a computational bottleneck. In this work, we develop a scalable computational approach for reducing the costs associated with the sampling stage of the BAE approach. Specifically, we consider the Taylor expansion of the accurate and surrogate forward models with respect to the uncertain parameter fields either as a control variate for variance reduction or as a means to efficiently approximate the mean and covariance of the approximation errors. We propose efficient methods for evaluating the expressions for the mean and covariance of the Taylor approximations based on linear(-ized) PDE solves. Furthermore, the proposed approach is independent of the dimension of the uncertain parameter, depending instead on the intrinsic dimension of the data, ensuring scalability to high-dimensional problems. The potential benefits of the proposed approach are demonstrated for two high-dimensional inverse problems governed by PDE examples, namely for the estimation of a distributed Robin boundary coefficient in a linear diffusion problem, and for a coefficient estimation problem governed by a nonlinear diffusion problem.

Figures (14)

• Figure 1: Example 1: Three prior samples of $b$.
• Figure 1: Example 1: Convergence of the mean of the approximation errors in the $\ell^2$-norm (left) and the $\ell^{\infty}$-norm (center) as well as the convergence of the covariance of the approximation errors (right). The errors for the standard MC, linear Taylor approximation control variable, and quadratic Taylor approximation control variable are shown in blue, red, and yellow respectively. Also shown is the asymptotic convergence rate $1/\sqrt{N}$ using a dashed line.
• Figure 1: Posterior estimates found when using the true auxiliary parameter value during inversion. For Example 1 (left) shown are is the MAP estimate in blue, the truth in red, two samples from the (Gaussian approximation to the) posterior in yellow, while the plus/minus one and two standard deviation intervals, $\pm\sigma$ and $\pm2\sigma$ are shaded in gray. For Example 2 we show the MAP estimate (centre) as well as the one-dimensional marginal posterior plots (right) along the line from (0,1) to (1,0) (see Figure \ref{['fig: MAPQPAT']}) with the MAP is shown in blue, the truth in shown in red, while two samples from the (Gaussian approximation to the) posterior are shown in yellow, and the plus/minus one and two standard deviation intervals, $\pm\sigma$ and $\pm2\sigma$ are shaded in gray.
• Figure 2: Example 1: The prior for the parameter $m_{\rm true}$ (left) with the true value $m_{\rm true}$ in red, two samples in yellow, the prior mean in blue and the $\pm\sigma_{\rm prior}$ and $\pm2\sigma_{\rm prior}$ intervals shaded. Also shown are the true auxiliary parameter $b_{\rm true}$ (centre) and the resulting solution to (\ref{['eq: Rob1']}) $u_{\rm true}$ (right) with measurement locations shown as black dots.
• Figure 2: Example 1: Convergence of the spectrum of $\boldsymbol{\Gamma}_\varepsilon$ using standard MC (left) linear CV (centre) and quadratic CV (right) using $N\in\{0,2,5,10,20,50,100,500,1000,5000,10000\}$ samples, dotted line is the noise level variance $\delta_e^2$.

Theorems & Definitions (3)

• Remark 1
• Remark 2
• Remark 3