Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey on Diffusion Models for Inverse Problems

Giannis Daras, Hyungjin Chung, Chieh-Hsin Lai, Yuki Mitsufuji, Jong Chul Ye, Peyman Milanfar, Alexandros G. Dimakis, Mauricio Delbracio

TL;DR

The survey addresses solving ill-posed inverse problems with pre-trained diffusion priors, focusing on unsupervised, training-free approaches. It organizes methods into four families—explicit score approximations, variational inference, CSGM-type latent-space strategies, and asymptotically exact sampling—and discusses conditional vs unconditional sampling, linear vs nonlinear degradations, and latent-diffusion challenges. It highlights practical considerations, such as measurement-model reliance, computational trade-offs, and the potential of ambient and blind settings, while offering a cohesive framework to connect seemingly disparate methods. The work serves as a reference point for researchers and engineers to understand the landscape, compare methods, and identify promising directions and benchmarks for diffusion-based inverse-problem solvers.

Abstract

Diffusion models have become increasingly popular for generative modeling due to their ability to generate high-quality samples. This has unlocked exciting new possibilities for solving inverse problems, especially in image restoration and reconstruction, by treating diffusion models as unsupervised priors. This survey provides a comprehensive overview of methods that utilize pre-trained diffusion models to solve inverse problems without requiring further training. We introduce taxonomies to categorize these methods based on both the problems they address and the techniques they employ. We analyze the connections between different approaches, offering insights into their practical implementation and highlighting important considerations. We further discuss specific challenges and potential solutions associated with using latent diffusion models for inverse problems. This work aims to be a valuable resource for those interested in learning about the intersection of diffusion models and inverse problems.

A Survey on Diffusion Models for Inverse Problems

TL;DR

The survey addresses solving ill-posed inverse problems with pre-trained diffusion priors, focusing on unsupervised, training-free approaches. It organizes methods into four families—explicit score approximations, variational inference, CSGM-type latent-space strategies, and asymptotically exact sampling—and discusses conditional vs unconditional sampling, linear vs nonlinear degradations, and latent-diffusion challenges. It highlights practical considerations, such as measurement-model reliance, computational trade-offs, and the potential of ambient and blind settings, while offering a cohesive framework to connect seemingly disparate methods. The work serves as a reference point for researchers and engineers to understand the landscape, compare methods, and identify promising directions and benchmarks for diffusion-based inverse-problem solvers.

Abstract

Diffusion models have become increasingly popular for generative modeling due to their ability to generate high-quality samples. This has unlocked exciting new possibilities for solving inverse problems, especially in image restoration and reconstruction, by treating diffusion models as unsupervised priors. This survey provides a comprehensive overview of methods that utilize pre-trained diffusion models to solve inverse problems without requiring further training. We introduce taxonomies to categorize these methods based on both the problems they address and the techniques they employ. We analyze the connections between different approaches, offering insights into their practical implementation and highlighting important considerations. We further discuss specific challenges and potential solutions associated with using latent diffusion models for inverse problems. This work aims to be a valuable resource for those interested in learning about the intersection of diffusion models and inverse problems.
Paper Structure (75 sections, 5 theorems, 114 equations, 1 figure, 1 table)

This paper contains 75 sections, 5 theorems, 114 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Problem Setting
  3. Recovery types
  4. Approaches for Solving Inverse Problems
  5. Unsupervised methods.
  6. Background
  7. Diffusion Processes
  8. Forward and Reverse Processes.
  9. Probability Flow ODE.
  10. SDE variants: Variance Exploding and Variance Preserving Processes.
  11. Tweedie's Formula and Denoising Score Matching
  12. Latent Diffusion Processes
  13. Conditional Sampling
  14. Stochastic Samplers for Inverse Problems
  15. Deterministic Samplers for Inverse Problems
  16. ...and 60 more sections

Key Result

Lemma A.1

Let ${\bm{X}}_0$ and ${\bm{X}}_t$ be two random variables, and $h_{\bm{\theta}}({\bm{x}}_t, t)$ be a function parameterized by $\bm{\theta}$. Then: That is, the function $h_{\bm{\theta}}({\bm{x}}_t, t)$ that minimizes the mean squared error with respect to ${\bm{x}}_0$ is the one that best approximates the conditional expectation $\mathbb{E}[{\bm{x}}_0|{\bm{x}}_t]$.

Figures (1)

  • Figure 1: Approximations for the measurements score proposed by different methods.

Theorems & Definitions (15)

  • Example 1.1: Denoising
  • Example 1.2: Inpainting
  • Example 1.3: Compressed Sensing
  • Example 1.4: Convolutions
  • Example 1.5: Phase Retrieval fienup1982phase
  • Example 1.6: Compression Removal
  • Lemma A.1: Conditional Expectation and MMSE
  • proof
  • Lemma A.2: Tweedie's Formula
  • proof
  • ...and 5 more