Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving Inverse Problems with Score-Based Generative Priors learned from Noisy Data

Asad Aali, Marius Arvinte, Sidharth Kumar, Jonathan I. Tamir

TL;DR

This work addresses learning score-based priors from noisy data when clean training samples are unavailable, introducing SURE-Score to jointly denoise and learn the score function using Stein's unbiased risk estimate. By coupling SURE with denoising score matching through Tweedie's rule, the method reuses a single network to perform MMSE denoising and score learning, enabling posterior sampling for inverse problems. The authors demonstrate its effectiveness on MIMO channel estimation and accelerated MRI, showing competitive performance even when training data are at $0$ dB SNR and that self-supervised denoising approaches can approach supervised baselines. The approach broadens the practicality of score-based priors in real-world, noisy-data regimes and suggests extensions to other noise models and domains.

Abstract

We present SURE-Score: an approach for learning score-based generative models using training samples corrupted by additive Gaussian noise. When a large training set of clean samples is available, solving inverse problems via score-based (diffusion) generative models trained on the underlying fully-sampled data distribution has recently been shown to outperform end-to-end supervised deep learning. In practice, such a large collection of training data may be prohibitively expensive to acquire in the first place. In this work, we present an approach for approximately learning a score-based generative model of the clean distribution, from noisy training data. We formulate and justify a novel loss function that leverages Stein's unbiased risk estimate to jointly denoise the data and learn the score function via denoising score matching, while using only the noisy samples. We demonstrate the generality of SURE-Score by learning priors and applying posterior sampling to ill-posed inverse problems in two practical applications from different domains: compressive wireless multiple-input multiple-output channel estimation and accelerated 2D multi-coil magnetic resonance imaging reconstruction, where we demonstrate competitive reconstruction performance when learning at signal-to-noise ratio values of 0 and 10 dB, respectively.

Solving Inverse Problems with Score-Based Generative Priors learned from Noisy Data

TL;DR

This work addresses learning score-based priors from noisy data when clean training samples are unavailable, introducing SURE-Score to jointly denoise and learn the score function using Stein's unbiased risk estimate. By coupling SURE with denoising score matching through Tweedie's rule, the method reuses a single network to perform MMSE denoising and score learning, enabling posterior sampling for inverse problems. The authors demonstrate its effectiveness on MIMO channel estimation and accelerated MRI, showing competitive performance even when training data are at dB SNR and that self-supervised denoising approaches can approach supervised baselines. The approach broadens the practicality of score-based priors in real-world, noisy-data regimes and suggests extensions to other noise models and domains.

Abstract

We present SURE-Score: an approach for learning score-based generative models using training samples corrupted by additive Gaussian noise. When a large training set of clean samples is available, solving inverse problems via score-based (diffusion) generative models trained on the underlying fully-sampled data distribution has recently been shown to outperform end-to-end supervised deep learning. In practice, such a large collection of training data may be prohibitively expensive to acquire in the first place. In this work, we present an approach for approximately learning a score-based generative model of the clean distribution, from noisy training data. We formulate and justify a novel loss function that leverages Stein's unbiased risk estimate to jointly denoise the data and learn the score function via denoising score matching, while using only the noisy samples. We demonstrate the generality of SURE-Score by learning priors and applying posterior sampling to ill-posed inverse problems in two practical applications from different domains: compressive wireless multiple-input multiple-output channel estimation and accelerated 2D multi-coil magnetic resonance imaging reconstruction, where we demonstrate competitive reconstruction performance when learning at signal-to-noise ratio values of 0 and 10 dB, respectively.
Paper Structure (18 sections, 22 equations, 4 figures, 2 tables)

This paper contains 18 sections, 22 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contributions
  4. System Model and Background
  5. Linear Inverse Problems
  6. Score-Based Generative Modeling
  7. Solving Inverse Problems using Annealed Langevin Dynamics
  8. Stein's Unbiased Risk Estimate
  9. Proposed Method: SURE-Score
  10. Experimental Results and Discussion
  11. Model Architecture
  12. MIMO Channel Modeling and Estimation
  13. Prior Sampling
  14. Posterior Reconstruction
  15. Multi-Coil MRI Reconstruction
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Flow of SURE-Score during training. The same deep neural network $s_\theta$ is used first for denoising and subsequently for denoising score matching. The terms in \ref{['eq:our_loss_multicol2']} are obtained from the input, $\hat{x}_\mathrm{MMSE}$, and the final output.
  • Figure 2: Prior sampling performance for three methods: naive (ii), SURE-Score at SNR$^w = 0$ dB, and supervised (i). Each column shows a different realization of CDL-C channels.
  • Figure 3: Channel estimation performance at $\alpha = 0.6$ ($38$ pilots) using score models trained on CDL-C channels at SNR$^w$: $0$ dB (top) and $10$ dB (bottom).
  • Figure 4: Multi-coil MRI reconstruction at acceleration factor of $5\times$. From left to right: fully sampled ground truth, linear reconstruction, posterior sampling after naively training on noisy data at $\mathrm{SNR}^w$, posterior sampling after training with SURE-Score, and posterior sampling after training with noise-free data. The bottom row shows the sampling pattern and difference images for each method, respectively.