Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conditional Generative Models are Provably Robust: Pointwise Guarantees for Bayesian Inverse Problems

Fabian Altekrüger, Paul Hagemann, Gabriele Steidl

TL;DR

This work proves a pointwise robustness guarantee for conditional generative models in Bayesian inverse problems: if a conditional generator achieves a small Wasserstein-1 training loss, then for a given observation $\tilde{y}$ the generated posterior sample distribution remains close to the true posterior in $W_1$, with a bound scaling as $\varepsilon^{1/(n+1)}$. The authors develop the theoretical framework by establishing local Lipschitz continuity for both the generator and the posterior, and then demonstrate how major conditional models—Conditional Normalizing Flows, Conditional Wasserstein GANs, Conditional Diffusion Models, and Conditional VAEs—can achieve the required small $W_1$ discrepancy through their respective training losses. The results connect standard training objectives (KL, Wasserstein, score matching, ELBO) to explicit pointwise posterior stability, providing guidance on model choice and highlighting limitations for out-of-distribution observations. Practically, the findings offer a principled basis for robust posterior sampling from single measurements in inverse problems, and point to further work on tightening constants and extending to KL-based or adversarial settings.

Abstract

Conditional generative models became a very powerful tool to sample from Bayesian inverse problem posteriors. It is well-known in classical Bayesian literature that posterior measures are quite robust with respect to perturbations of both the prior measure and the negative log-likelihood, which includes perturbations of the observations. However, to the best of our knowledge, the robustness of conditional generative models with respect to perturbations of the observations has not been investigated yet. In this paper, we prove for the first time that appropriately learned conditional generative models provide robust results for single observations.

Conditional Generative Models are Provably Robust: Pointwise Guarantees for Bayesian Inverse Problems

TL;DR

This work proves a pointwise robustness guarantee for conditional generative models in Bayesian inverse problems: if a conditional generator achieves a small Wasserstein-1 training loss, then for a given observation the generated posterior sample distribution remains close to the true posterior in , with a bound scaling as . The authors develop the theoretical framework by establishing local Lipschitz continuity for both the generator and the posterior, and then demonstrate how major conditional models—Conditional Normalizing Flows, Conditional Wasserstein GANs, Conditional Diffusion Models, and Conditional VAEs—can achieve the required small discrepancy through their respective training losses. The results connect standard training objectives (KL, Wasserstein, score matching, ELBO) to explicit pointwise posterior stability, providing guidance on model choice and highlighting limitations for out-of-distribution observations. Practically, the findings offer a principled basis for robust posterior sampling from single measurements in inverse problems, and point to further work on tightening constants and extending to KL-based or adversarial settings.

Abstract

Conditional generative models became a very powerful tool to sample from Bayesian inverse problem posteriors. It is well-known in classical Bayesian literature that posterior measures are quite robust with respect to perturbations of both the prior measure and the negative log-likelihood, which includes perturbations of the observations. However, to the best of our knowledge, the robustness of conditional generative models with respect to perturbations of the observations has not been investigated yet. In this paper, we prove for the first time that appropriately learned conditional generative models provide robust results for single observations.
Paper Structure (12 sections, 7 theorems, 63 equations, 4 figures)

This paper contains 12 sections, 7 theorems, 63 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Outline of the paper.
  3. Pointwise Robustness of Conditional Generative NNs
  4. Conditional Generative Models
  5. Conditional Normalizing Flows
  6. Conditional Wasserstein GANs
  7. Conditional Diffusion Models
  8. Conditional Variational Autoencoder
  9. Conclusion
  10. Example on the Robustness of the MAP and Posterior
  11. Local Lipschitz continuity of the generator for a latent space with infinite support
  12. Experimental demonstration of assumption \ref{['eq:expectation_W1']}

Key Result

Lemma 1

For all $r>0$, there exists some $L_r >0$ such that for any parameterized family of generative models $G$ with $\Vert \nabla_y G(y,z) \Vert \leq L_r$ for all $z \in \mathrm{supp}(P_Z)$ and all $y \in \mathbb{R}^n$ with $\Vert y \Vert \le r$ it holds for all $y_1,y_2 \in \mathbb{R}^n$ with $\Vert y_1 \Vert, \Vert y_2 \Vert \le r$.

Figures (4)

  • Figure 1: Posterior density (red), MAP estimator (blue) and MMSE estimator (green) for different observations $y=-0.05,-0.01,0.01,0.05$ (from left to right). While the MAP estimator is discontinuous with respect to the observation $y$, the posterior density is continuous with respect to y. The MMSE estimator just gives the expectation value of the posterior which is, in contrast to MAP, not the value with highest probability.
  • Figure 2: Geometric interpretation of the proof of Theorem \ref{['thm:posterior_bound']}. Inside the ball $B_r(\tilde{y})$ we can find some $\hat{y}$, for which the Wasserstein distance $W_1 (P_{X|Y=\hat{y}},G(\hat{y},\cdot)_{\#}P_Z)$ is bounded by $\frac{2\varepsilon}{S_n r^n a}$. Using the regularity of the generator and of the inverse problem, the Wasserstein distance $W_1 (P_{X|Y=\tilde{y}},G(\tilde{y},\cdot)_{\#}P_Z)$ can be bounded by the triangle inequality.
  • Figure 3: The MAP estimator (top) and the MMSE estimator (bottom) with respect to the observation $y$ for $\varepsilon^2 = 0.05^2$ and different noise levels $\sigma^2$.
  • Figure 4: Expectation of the Wasserstein distance between posterior and pushforward of the generator with respect to the training loss of the conditional normalizing flow.

Theorems & Definitions (15)

  • Lemma 1: Local Lipschitz continuity of generator
  • proof
  • Remark 2
  • Lemma 3: Local Lipschitz continuity of the posterior
  • Remark 4
  • Theorem 5
  • proof
  • Remark 6
  • Corollary 7
  • proof
  • ...and 5 more