Measurement Score-Based Diffusion Model

TL;DR

MSM tackles the challenge of diffusion-model training without clean ground-truth data by learning partial measurement scores from noisy, subsampled measurements and modeling the full measurement distribution as an expectation over randomized subsampling. It introduces a stochastic, minibatch-based MSM score estimator and a sampling procedure for unconditional generation, plus a posterior sampling mechanism for inverse problems, with a Girsanov-based KL bound showing how minibatch size controls accuracy. Theoretical results bound $D_{\mathsf{KL}}(q(\bm{z}) \| \widehat{q}(\bm{z})) \le \frac{v^2}{w} C$ under a bounded-variance assumption, and experiments on RGB faces and multi-coil MRI demonstrate state-of-the-art performance among diffusion methods trained without clean data. Overall, MSM enables high-quality image synthesis and reliable inverse-problem solutions in data-limited regimes, with broad applicability to MRI and other modalities; code is available at the authors’ repository.

Abstract

Diffusion models are widely used in applications ranging from image generation to inverse problems. However, training diffusion models typically requires clean ground-truth images, which are unavailable in many applications. We introduce the Measurement Score-based diffusion Model (MSM), a novel framework that learns partial measurement scores using only noisy and subsampled measurements. MSM models the distribution of full measurements as an expectation over partial scores induced by randomized subsampling. To make the MSM representation computationally efficient, we also develop a stochastic sampling algorithm that generates full images by using a randomly selected subset of partial scores at each step. We additionally propose a new posterior sampling method for solving inverse problems that reconstructs images using these partial scores. We provide a theoretical analysis that bounds the Kullback-Leibler divergence between the distributions induced by full and stochastic sampling, establishing the accuracy of the proposed algorithm. We demonstrate the effectiveness of MSM on natural images and multi-coil MRI, showing that it can generate high-quality images and solve inverse problems -- all without access to clean training data. Code is available at https://github.com/wustl-cig/MSM.

Figures (8)

• Figure 1: Illustration of the Measurement Score-based diffusion Model (MSM) for training and sampling using subsampled data. Training: MSM is trained solely on degraded measurements. Diffusion noise is added to these measurements, and the model learns to denoise them. Sampling: At each diffusion step, MSM randomly subsamples the current full-measurement iterate, denoises the resulting partial measurement, and aggregates multiple outputs. A weighting vector compensates for overlapping contributions across partial measurements. See Figure \ref{['fig:illustration_mri']} for the MRI-specific version.
• Figure 2: Generated samples from MSM trained under three degradation settings (first row: training data; second row: samples generated by models trained on the corresponding data). Note how despite never seeing ground-truth data, MSM can generated high-quality images.
• Figure 3: Visual comparison of methods trained on subsampled data for inverse problems. Note how MSM leads to the best results in both applications.
• Figure 4: Illustration of the Measurement Score-based diffusion Model (MSM) for training and sampling using subsampled MRI measurements. MSM operates directly on k-space measurements, with minor domain transformations between k-space and image space only at the input and output of the diffusion model $\mathsf{D}_{\theta}$.
• Figure 5: Visual comparison of MSM trained under extreme subsampling ($R = 8$) with MSM and baseline methods trained under less degraded conditions.

Theorems & Definitions (7)

• Theorem 1
• Theorem
• proof
• Lemma 1
• Lemma 2
• proof
• Lemma 3