Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GSURE-Based Diffusion Model Training with Corrupted Data

Bahjat Kawar, Noam Elata, Tomer Michaeli, Michael Elad

TL;DR

This work proposes a novel training technique for generative diffusion models based only on corrupted data and introduces a loss function based on the Generalized Stein's Unbiased Risk Estimator (GSURE), and proves that under some conditions, it is equivalent to the training objective used in fully supervised diffusion models.

Abstract

Diffusion models have demonstrated impressive results in both data generation and downstream tasks such as inverse problems, text-based editing, classification, and more. However, training such models usually requires large amounts of clean signals which are often difficult or impossible to obtain. In this work, we propose a novel training technique for generative diffusion models based only on corrupted data. We introduce a loss function based on the Generalized Stein's Unbiased Risk Estimator (GSURE), and prove that under some conditions, it is equivalent to the training objective used in fully supervised diffusion models. We demonstrate our technique on face images as well as Magnetic Resonance Imaging (MRI), where the use of undersampled data significantly alleviates data collection costs. Our approach achieves generative performance comparable to its fully supervised counterpart without training on any clean signals. In addition, we deploy the resulting diffusion model in various downstream tasks beyond the degradation present in the training set, showcasing promising results.

GSURE-Based Diffusion Model Training with Corrupted Data

TL;DR

This work proposes a novel training technique for generative diffusion models based only on corrupted data and introduces a loss function based on the Generalized Stein's Unbiased Risk Estimator (GSURE), and proves that under some conditions, it is equivalent to the training objective used in fully supervised diffusion models.

Abstract

Diffusion models have demonstrated impressive results in both data generation and downstream tasks such as inverse problems, text-based editing, classification, and more. However, training such models usually requires large amounts of clean signals which are often difficult or impossible to obtain. In this work, we propose a novel training technique for generative diffusion models based only on corrupted data. We introduce a loss function based on the Generalized Stein's Unbiased Risk Estimator (GSURE), and prove that under some conditions, it is equivalent to the training objective used in fully supervised diffusion models. We demonstrate our technique on face images as well as Magnetic Resonance Imaging (MRI), where the use of undersampled data significantly alleviates data collection costs. Our approach achieves generative performance comparable to its fully supervised counterpart without training on any clean signals. In addition, we deploy the resulting diffusion model in various downstream tasks beyond the degradation present in the training set, showcasing promising results.
Paper Structure (28 sections, 4 theorems, 21 equations, 9 figures, 3 tables, 1 algorithm)

This paper contains 28 sections, 4 theorems, 21 equations, 9 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Denoising Diffusion Probabilistic Models
  4. Generalized Stein's Unbiased Risk Estimator (GSURE)
  5. GSURE-Diffusion: Mathematical Formulation
  6. Problem Formulation
  7. GSURE-Based Denoising Diffusion Loss Function
  8. Experiments
  9. Human Face Images
  10. Magnetic Resonance Imaging (MRI)
  11. Limitations
  12. Related Work
  13. Conclusion
  14. Proposition Proofs
  15. Detailed Data Descriptions
  16. ...and 13 more sections

Key Result

Proposition 3.0

For ${\mathbf{x}} \sim q({\mathbf{x}})$, $\bar{\mathbf{x}} = {\bm{V}}^\top {\mathbf{x}}$, $\bar{\mathbf{x}}_t$ sampled from eq:xbart-marginal, and the diagonal weight matrix ${\bm{W}} = \mathbb{E}[{\bm{P}}]^{-\frac{1}{2}} \succ 0$ (positive definite), if ${\bm{P}}$ and $\left(f_\theta^{(t)}(\bar{\ma

Figures (9)

  • Figure 1: Training sets samples of the different degradation settings in CelebA liu2015celeba experiments.
  • Figure 2: Generated samples (with $50$ DDIM song2020denoising steps) from models trained on different degradation settings in CelebA liu2015celeba experiments.
  • Figure 3: Accelerated MRI reconstruction results for $R=4$ and $\sigma_0=0.01$.
  • Figure 4: Left: Denoising MSE (on fully sampled noisy images) for the GSURE-Diffusion and oracle models across diffusion timesteps. Right: Qualitative denoising examples for both models.
  • Figure 5: Accelerated MRI reconstruction results for $R \in \{6,8,10,12\}$ and $\sigma_0=0.01$. GSURE-Diffusion can generalize well across different acceleration factors.
  • ...and 4 more figures

Theorems & Definitions (6)

  • Proposition 3.0
  • Proposition 3.0
  • Proposition A.0
  • proof
  • Proposition A.0
  • proof