Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Analyzing and Guiding Zero-Shot Posterior Sampling in Diffusion Models

Roi Benita, Michael Elad, Joseph Keshet

TL;DR

This work analyzes zero-shot posterior sampling in diffusion models for linear inverse problems through a spectral lens under a Gaussian prior. It derives closed-form expressions for both the ideal posterior sampler and training-free reconstruction methods, enabling principled comparisons in the spectral domain and a method-agnostic framework for weighting guidance terms. The authors formulate an optimization based on averaged Wasserstein distance to design optimal guidance weights, and provide closed-form, per-frequency transfer functions that decouple across frequencies. Empirical results on synthetic Gaussian data and real datasets (FFHQ and ImageNet) show that the spectral recommendations offer a more balanced trade-off between measurement fidelity and perceptual quality than common heuristics, while reducing per-instance tuning requirements and adapting to diffusion step size.

Abstract

Recovering a signal from its degraded measurements is a long standing challenge in science and engineering. Recently, zero-shot diffusion based methods have been proposed for such inverse problems, offering a posterior sampling based solution that leverages prior knowledge. Such algorithms incorporate the observations through inference, often leaning on manual tuning and heuristics. In this work we propose a rigorous analysis of such approximate posterior-samplers, relying on a Gaussianity assumption of the prior. Under this regime, we show that both the ideal posterior sampler and diffusion-based reconstruction algorithms can be expressed in closed-form, enabling their thorough analysis and comparisons in the spectral domain. Building on these representations, we also introduce a principled framework for parameter design, replacing heuristic selection strategies used to date. The proposed approach is method-agnostic and yields tailored parameter choices for each algorithm, jointly accounting for the characteristics of the prior, the degraded signal, and the diffusion dynamics. We show that our spectral recommendations differ structurally from standard heuristics and vary with the diffusion step size, resulting in a consistent balance between perceptual quality and signal fidelity.

Analyzing and Guiding Zero-Shot Posterior Sampling in Diffusion Models

TL;DR

This work analyzes zero-shot posterior sampling in diffusion models for linear inverse problems through a spectral lens under a Gaussian prior. It derives closed-form expressions for both the ideal posterior sampler and training-free reconstruction methods, enabling principled comparisons in the spectral domain and a method-agnostic framework for weighting guidance terms. The authors formulate an optimization based on averaged Wasserstein distance to design optimal guidance weights, and provide closed-form, per-frequency transfer functions that decouple across frequencies. Empirical results on synthetic Gaussian data and real datasets (FFHQ and ImageNet) show that the spectral recommendations offer a more balanced trade-off between measurement fidelity and perceptual quality than common heuristics, while reducing per-instance tuning requirements and adapting to diffusion step size.

Abstract

Recovering a signal from its degraded measurements is a long standing challenge in science and engineering. Recently, zero-shot diffusion based methods have been proposed for such inverse problems, offering a posterior sampling based solution that leverages prior knowledge. Such algorithms incorporate the observations through inference, often leaning on manual tuning and heuristics. In this work we propose a rigorous analysis of such approximate posterior-samplers, relying on a Gaussianity assumption of the prior. Under this regime, we show that both the ideal posterior sampler and diffusion-based reconstruction algorithms can be expressed in closed-form, enabling their thorough analysis and comparisons in the spectral domain. Building on these representations, we also introduce a principled framework for parameter design, replacing heuristic selection strategies used to date. The proposed approach is method-agnostic and yields tailored parameter choices for each algorithm, jointly accounting for the characteristics of the prior, the degraded signal, and the diffusion dynamics. We show that our spectral recommendations differ structurally from standard heuristics and vary with the diffusion step size, resulting in a consistent balance between perceptual quality and signal fidelity.
Paper Structure (37 sections, 3 theorems, 180 equations, 12 figures, 5 tables)

This paper contains 37 sections, 3 theorems, 180 equations, 12 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Linear Inverse Problems
  4. Diffusion Models
  5. training-free Diffusion-based methods for inverse problems
  6. Guided zero-shot posterior samplers
  7. The Optimal Denoiser for a Gaussian Prior
  8. Reverse Process Formulation
  9. Migrating to spectral domain
  10. Optimal Posterior Sampling Weights
  11. Optimal Gaussian Posterior Sampling
  12. The Posterior Optimal Denoiser
  13. Reverse Process Formulation
  14. Migrating to spectral domain
  15. Related work
  16. ...and 22 more sections

Key Result

Theorem 4.1

Let $\mathbf{x}_0 \sim \mathcal{N}(\boldsymbol{\mu}_0, \boldsymbol{\Sigma}_0)$ and let $\mathbf{x}_t$ denote the noisy signal obtained from the forward diffusion process, as defined in eq:marginal_dist. Given the linear measurement model $\mathbf{y}=\mathbf{H}\mathbf{x}_0+\mathbf{n}$, the denoised s and admits the following closed-form: A detailed proof is given in Appendix sec:appendix_Map_est.

Figures (12)

  • Figure 1: Spectral recommendations for the weighting coefficients $\boldsymbol{\zeta}$ in DPS for different numbers of diffusion steps $S\in[5, 30, 50, 70, 100, 120]$, with variability across realizations.
  • Figure 2: Comparison between the spectral recommendations (red) and DPS weighting coefficients for $70$ diffusion steps, evaluated for different heuristic values of $\zeta' \in \{0.1, 0.3, 0.5, 0.7, 1.0\}.$
  • Figure 3: Comparison of the Wasserstein-2 distance for DPS heuristic values $\zeta' \in \{0.1, 0.3, 0.5, 0.7, 1.0\}$, the spectral recommendations applied to DPS (red) and to $\Pi\text{GDM}$ (brown, dotted), and the analytically derived ideal posterior sampler (black), across different number of diffusion steps $S \in \{5, 10, 15, 20, 30, 50, 70, 100, 120, 150\}$.
  • Figure 4: Comparison of spectral recommendations on the FFHQ dataset, with a zoomed-in view of the DPS heuristics. Results are shown for selected diffusion steps $S\in\{30, 70, 100,150\}$.
  • Figure 5: Covariance matrix obtained for $d=50$ and $l=0.05$.
  • ...and 7 more figures

Theorems & Definitions (3)

  • Theorem 4.1
  • Lemma 4.2
  • Lemma 4.3