Error Analysis of Bayesian Inverse Problems with Generative Priors

TL;DR

An analysis for inverse problems involving generative models is presented by presenting quantitative error bounds for minimum Wasserstein-2 generative models for the prior and it is shown that under some assumptions, the error in the posterior due to the generative prior will inherit the same rate as the prior with respect to the Wasserstein-1 distance.

Abstract

Data-driven methods for the solution of inverse problems have become widely popular in recent years thanks to the rise of machine learning techniques. A popular approach concerns the training of a generative model on additional data to learn a bespoke prior for the problem at hand. In this article we present an analysis for such problems by presenting quantitative error bounds for minimum Wasserstein-2 generative models for the prior. We show that under some assumptions, the error in the posterior due to the generative prior will inherit the same rate as the prior with respect to the Wasserstein-1 distance. We further present numerical experiments that verify that aspects of our error analysis manifests in some benchmarks followed by an elliptic PDE inverse problem where a generative prior is used to model a non-stationary field.

Figures (8)

• Figure 1: Example of 2D Benchmarks from \ref{['sec:Toy Example']} with three example distributions. From left to right, the first column depicts the true prior $\mu$, the second column shows the data-driven prior $\widehat{\mu}$, the third column shows the true posterior samples $\nu$, while the fourth column shows samples from the data-driven posterior $\widehat{\nu}$. The heatmaps for the posterior were generated by reweighting the prior samples using the likelihood.
• Figure 2: Comparing the convergence of data-driven priors in the ${\mathcal{W}}_2$ metric versus the subsequent posteriors in the ${\mathcal{W}}_1$ metric as a function of the training sample size for the 2D benchmarks from \ref{['sec:Toy Example']}. In all cases the prior ${\mathcal{W}}_2$ distance controls the posterior ${\mathcal{W}}_1$ distance. Slopes of linear fits to these curves, indicating rate of convergence with the sample size, are reported in \ref{['tab:WDslopes']}.
• Figure 3: Comparing the convergence of data-driven priors in the ${\mathcal{W}}_2$ metric versus the subsequent posteriors in the ${\mathcal{W}}_1$ metric as a function of the width of the generator for the 2D benchmarks from \ref{['sec:Toy Example']}. In all cases the prior ${\mathcal{W}}_2$ distance controls the posterior ${\mathcal{W}}_1$ distance. Slopes of linear fits to these curves, indicating rate of convergence with width of the generator, are reported in \ref{['tab:WDslopes']}.
• Figure 4: Comparing the convergence of data-driven priors in the ${\mathcal{W}}_2$ metric versus the subsequent posteriors in the ${\mathcal{W}}_1$ metric as a function of the number of training epochs for the 2D benchmarks from \ref{['sec:Toy Example']}. In all cases the prior ${\mathcal{W}}_2$ distance controls the posterior ${\mathcal{W}}_1$ distance. Slopes of linear fits to these curves, indicating rate of convergence with the number of training epochs, are reported in \ref{['tab:WDslopes']}.
• Figure 5: Results of the Darcy flow experiments with $20\%$ noise-to-signal ratio: (a) the ground truth parameter; (b) posterior mean in the image space; (c) pointwise posterior standard deviation in the image space; (d) coordinate-wise autocorrelation functions (ACF) of the MCMC chain in the latent space; (e) coordinate-wise effective sample size (ESS) of the MCMC chain in the latent space; (f) the data $y$ plotted as the noisy pointwise observations of the pressure field $p$ arising from the true field in panel (a).

Theorems & Definitions (32)

• Proposition 2.1
• Theorem 2.2
• Proof 1
• Corollary 2.3
• Proof 2
• Remark 2.4
• Lemma 3.1
• Proof 3
• Remark 3.2
• Lemma 3.3: fournier2015rate