The Gaussian Latent Machine: Efficient Prior and Posterior Sampling for Inverse Problems

TL;DR

The paper introduces Gaussian latent machines (GLMs) to efficiently sample from product-of-experts priors and posteriors in Bayesian imaging. By lifting a PoE density to a joint latent model, GLMs yield a two-block Gibbs sampler in general, with $f_{X|Z}$ Gaussian and $f_{Z|X}$ decomposable into univariate conditionals, and enable direct sampling in complete models. The framework robustly handles improper priors and scales to high-dimensional imaging problems, demonstrated through extensive priors- and posteriors-based experiments that show rapid convergence, competitive sampling efficiency, and improved reconstruction when priors are learned. The work also outlines nonlinear extensions, direct-sampling strategies for complete models, and future directions, underscoring its practical impact for uncertainty-aware Bayesian imaging and principled prior design.

Abstract

We consider the problem of sampling from a product-of-experts-type model that encompasses many standard prior and posterior distributions commonly found in Bayesian imaging. We show that this model can be easily lifted into a novel latent variable model, which we refer to as a Gaussian latent machine. This leads to a general sampling approach that unifies and generalizes many existing sampling algorithms in the literature. Most notably, it yields a highly efficient and effective two-block Gibbs sampling approach in the general case, while also specializing to direct sampling algorithms in particular cases. Finally, we present detailed numerical experiments that demonstrate the efficiency and effectiveness of our proposed sampling approach across a wide range of prior and posterior sampling problems from Bayesian imaging.

Figures (17)

• Figure 1: Contour plots that illustrate (the absence of) closure properties of overcomplete models: In the Gaussian case (a), the overcomplete model can be equivalently represented by a complete model. This does not hold for the Laplacian case (b).
• Figure 1: Factor graph topologies used in the baseline experiments. Each node defines a scalar random variable and each factor is of the same functional form $\phi$. The pairwise factors in (c) and (d) are of the form $\phi_{ij}(*){x_i, x_j} := \phi(*){x_j - x_i}$ and are defined for edges $ij$ such that $i < j$.
• Figure 2: Markov network of the \ref{['def_glm']} model and the corresponding Gibbs sampling subproblems. Sampling $X \mid Z = z$ reduces to sampling from a multivariate Gaussian distribution, while sampling $Z \mid x = x$ reduces to sampling from $m$ univariate distributions.
• Figure 2: Ground-truth negative $\log$ marginals in the baseline experiments for the stationary distribution for the factor and product topologies, the edge marginal for the loop topology, and the inner edge marginal in the grid topology, and the outer edge marginal in the grid topology.
• Figure 3: $\log_{10}$ Wasserstein-1 distances of Gibbs (solid, ) and (dashed, ) over iterations in the baseline experiments for the factor and product topologies, the edge marginal of the loop topology, and the inner edge marginal of the grid topology and the outer edge marginal of the grid topology.

Theorems & Definitions (55)

• Definition 1.1: Factor marginalization property
• Proposition 2.1
• Proposition 2.2
• Remark 2.3
• Remark 2.4
• Proposition 2.5
• Remark 2.6
• Remark 2.7
• Remark 2.8
• Corollary 2.9