Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Non-asymptotic error estimates for the Laplace approximation in Bayesian inverse problems

Tapio Helin, Remo Kretschmann

TL;DR

This work provides non-asymptotic error estimates for the Laplace approximation of posteriors in nonlinear Bayesian inverse problems, quantified via total variation distance. It derives a central two-term bound that splits error into near- and far-field contributions and identifies an optimal radius for balancing these terms, under assumptions on third-differentials and a quadratic lower bound. It further gives explicit, dimension-aware bounds and analyzes asymptotic regimes for fixed and increasing dimension, as well as a perturbation theory for perturbed linear forward maps with Gaussian priors, showing linear-in-perturbation convergence of the Laplace error. The results offer computable guidance on when Laplace-based approximations are reliable in nonlinear inverse problems and shed light on how nonlinearity and dimension influence the accuracy of Gaussian approximations in Bayesian inference.

Abstract

In this paper we study properties of the Laplace approximation of the posterior distribution arising in nonlinear Bayesian inverse problems. Our work is motivated by Schillings et al. (2020), where it is shown that in such a setting the Laplace approximation error in Hellinger distance converges to zero in the order of the noise level. Here, we prove novel error estimates for a given noise level that also quantify the effect due to the nonlinearity of the forward mapping and the dimension of the problem. In particular, we are interested in settings in which a linear forward mapping is perturbed by a small nonlinear mapping. Our results indicate that in this case, the Laplace approximation error is of the size of the perturbation. The paper provides insight into Bayesian inference in nonlinear inverse problems, where linearization of the forward mapping has suitable approximation properties.

Non-asymptotic error estimates for the Laplace approximation in Bayesian inverse problems

TL;DR

This work provides non-asymptotic error estimates for the Laplace approximation of posteriors in nonlinear Bayesian inverse problems, quantified via total variation distance. It derives a central two-term bound that splits error into near- and far-field contributions and identifies an optimal radius for balancing these terms, under assumptions on third-differentials and a quadratic lower bound. It further gives explicit, dimension-aware bounds and analyzes asymptotic regimes for fixed and increasing dimension, as well as a perturbation theory for perturbed linear forward maps with Gaussian priors, showing linear-in-perturbation convergence of the Laplace error. The results offer computable guidance on when Laplace-based approximations are reliable in nonlinear inverse problems and shed light on how nonlinearity and dimension influence the accuracy of Gaussian approximations in Bayesian inference.

Abstract

In this paper we study properties of the Laplace approximation of the posterior distribution arising in nonlinear Bayesian inverse problems. Our work is motivated by Schillings et al. (2020), where it is shown that in such a setting the Laplace approximation error in Hellinger distance converges to zero in the order of the noise level. Here, we prove novel error estimates for a given noise level that also quantify the effect due to the nonlinearity of the forward mapping and the dimension of the problem. In particular, we are interested in settings in which a linear forward mapping is perturbed by a small nonlinear mapping. Our results indicate that in this case, the Laplace approximation error is of the size of the perturbation. The paper provides insight into Bayesian inference in nonlinear inverse problems, where linearization of the forward mapping has suitable approximation properties.

Paper Structure

This paper contains 16 sections, 15 theorems, 125 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Our contributions
  3. Relevant literature
  4. Organization of the paper
  5. Preliminaries and set-up
  6. Central error estimate
  7. Proof of Proposition \ref{['close_range_est']}
  8. Proof of Proposition \ref{['far_range_est']}
  9. Optimal choice of the parameter
  10. Explicit error estimate
  11. Asymptotic behavior for fixed and increasing problem dimension
  12. Perturbed linear problems with Gaussian prior
  13. Verifying Assumption \ref{['quadratic_bound_I']}
  14. Verifying Assumption \ref{['bound_third_diff']}
  15. Proof of Theorem \ref{['conv_rate_nonlin_perturb']}
  16. ...and 1 more sections

Key Result

Theorem 3.4

Suppose that unique_minimizerbound_third_diffquadratic_bound_I hold. Then we have for all $r_0 \ge 0$, where for all $r_0 \ge 0$, for all $r \ge 0$, and

Figures (1)

  • Figure 1: The probability densities of a posterior distribution $\mu^y$ and its Laplace approximation ${\mathcal{L}_{\mu^y}}$ (left), as well as the integrands of the total variation distance between $\mu^y$ and ${\mathcal{L}_{\mu^y}}$ and of the fundamental estimate \ref{['dTV_fund_int']} (right).

Theorems & Definitions (37)

  • Theorem 3.4
  • Remark 3.5
  • Proposition 3.6: Fundamental estimate
  • proof
  • Proposition 3.7: Close range estimate
  • Proposition 3.8: Far range estimate
  • proof : Proof of \ref{['main_estimate']}
  • Remark 3.9
  • Remark 3.10
  • Lemma 3.11
  • ...and 27 more