Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Posterior Consistency for Gaussian Process Approximations of Bayesian Posterior Distributions

Andrew M. Stuart, Aretha L. Teckentrup

TL;DR

This work proves error bounds on the Hellinger distance between the true posterior distribution and various approximations based on the Gaussian process emulator can be bounded by moments of the error in the emulator.

Abstract

We study the use of Gaussian process emulators to approximate the parameter-to-observation map or the negative log-likelihood in Bayesian inverse problems. We prove error bounds on the Hellinger distance between the true posterior distribution and various approximations based on the Gaussian process emulator. Our analysis includes approximations based on the mean of the predictive process, as well as approximations based on the full Gaussian process emulator. Our results show that the Hellinger distance between the true posterior and its approximations can be bounded by moments of the error in the emulator. Numerical results confirm our theoretical findings.

Posterior Consistency for Gaussian Process Approximations of Bayesian Posterior Distributions

TL;DR

This work proves error bounds on the Hellinger distance between the true posterior distribution and various approximations based on the Gaussian process emulator can be bounded by moments of the error in the emulator.

Abstract

We study the use of Gaussian process emulators to approximate the parameter-to-observation map or the negative log-likelihood in Bayesian inverse problems. We prove error bounds on the Hellinger distance between the true posterior distribution and various approximations based on the Gaussian process emulator. Our analysis includes approximations based on the mean of the predictive process, as well as approximations based on the full Gaussian process emulator. Our results show that the Hellinger distance between the true posterior and its approximations can be bounded by moments of the error in the emulator. Numerical results confirm our theoretical findings.

Paper Structure

This paper contains 16 sections, 18 theorems, 96 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Bayesian Inverse Problems
  3. Gaussian Process Regression
  4. Native spaces of Matèrn kernels
  5. Radial basis function interpolation
  6. Approximation of the Bayesian posterior distribution
  7. Approximation based on the predictive mean
  8. Approximations based on the predictive process
  9. Moment bounds on $Z_{N, \mathcal{G}}^\mathrm{sample}$ and $Z_{N, \Phi}^\mathrm{sample}$
  10. Error in the marginal approximations $\mu^{y,N,\mathcal{G}}_\mathrm{marginal}$ and $\mu^{y,N,\Phi}_\mathrm{marginal}$
  11. Error in the random approximations $\mu^{y,N,\mathcal{G}}_\mathrm{sample}$ and $\mu^{y,N,\Phi}_\mathrm{sample}$
  12. Numerical Examples
  13. Mean-based approximations
  14. Marginal approximations
  15. Random approximations
  16. ...and 1 more sections

Key Result

Proposition 2.1

(stuart10) Suppose $\mathcal{G} : { \mathbb{R}^K} \rightarrow \mathbb{R}^J$ is continuous. Then the posterior distribution $\mu^y$ on the conditioned random variable $u | y$ is absolutely continuous with respect to $\mu_0$ and given by Bayes' Theorem: where

Figures (4)

  • Figure 1: $2 d_{\hbox{\tiny{\rm Hell}}}(\mu^y, \mu^{y,N,\mathcal{G}}_\mathrm{mean})^2$ (left) and $2 d_{\hbox{\tiny{\rm Hell}}}(\mu^y, \mu^{y,N,\Phi}_\mathrm{mean})^2$ (right), for a variety of choices of $K$ and $\nu$, for $J=1$.
  • Figure 2: $2 d_{\hbox{\tiny{\rm Hell}}}(\mu^y, \mu^{y,N,\mathcal{G}}_\mathrm{mean})^2$ (left) and $2 d_{\hbox{\tiny{\rm Hell}}}(\mu^y, \mu^{y,N,\Phi}_\mathrm{mean})^2$ (right), for a variety of choices of $K$ and $\nu=1$, for $J=15$.
  • Figure 3: $2 d_{\hbox{\tiny{\rm Hell}}}(\mu^y, \mu^{y,N,\mathcal{G}}_\mathrm{marginal})^2$ (left) and $2 d_{\hbox{\tiny{\rm Hell}}}(\mu^y, \mu^{y,N,\Phi}_\mathrm{marginal})^2$ (right), for a variety of choices of $K$ and $\nu$, for $J=1$.
  • Figure 4: $2 \mathbb{E}_{\nu_N^\mathcal{G}} (d_{\hbox{\tiny{\rm Hell}}}(\mu^y, \mu^{y,N,\mathcal{G}}_\mathrm{sample})^2)$ (left) and $2 \mathbb{E}_{\nu_N^\Phi} (d_{\hbox{\tiny{\rm Hell}}}(\mu^y, \mu^{y,N,\Phi}_\mathrm{sample})^2)$ (right), for a variety of choices of $K$ and $\nu$, for $J=1$.

Theorems & Definitions (31)

  • Proposition 2.1
  • Definition 3.1
  • Proposition 3.2
  • Proposition 3.3
  • Proposition 3.4
  • Proposition 3.5
  • proof
  • Remark 3.6
  • Remark 3.7
  • Lemma 4.1
  • ...and 21 more