UNSURE: self-supervised learning with Unknown Noise level and Stein's Unbiased Risk Estimate

TL;DR

This work addresses learning image reconstructions from noisy measurements without ground-truth references by formalizing an expressivity–robustness trade-off and introducing UNSURE, a self-supervised objective that constrains the estimator's expected divergence to zero. It extends SURE-like losses to unknown noise levels and to broader noise models, including correlated Gaussian, Poisson–Gaussian, and exponential-family noises, as well as general inverse problems. The paper derives closed-form or tractable solutions for divergence-free estimators, introduces two practical implementations (Lagrange-based UNSURE and score-based UNSURE), and demonstrates state-of-the-art performance across MNIST denoising, color-noise DIV2K, CT, MRI, and real Cryo-EM data under unknown noise conditions. These results suggest UNSURE as a practical, robust alternative to supervised training in settings where noise statistics are uncertain or unavailable, with broad applicability to imaging inverse problems.

Abstract

Recently, many self-supervised learning methods for image reconstruction have been proposed that can learn from noisy data alone, bypassing the need for ground-truth references. Most existing methods cluster around two classes: i) Stein's Unbiased Risk Estimate (SURE) and similar approaches that assume full knowledge of the noise distribution, and ii) Noise2Self and similar cross-validation methods that require very mild knowledge about the noise distribution. The first class of methods tends to be impractical, as the noise level is often unknown in real-world applications, and the second class is often suboptimal compared to supervised learning. In this paper, we provide a theoretical framework that characterizes this expressivity-robustness trade-off and propose a new approach based on SURE, but unlike the standard SURE, does not require knowledge about the noise level. Throughout a series of experiments, we show that the proposed estimator outperforms other existing self-supervised methods on various imaging inverse problems.

Figures (6)

• Figure 1: The expressivity-robustness trade-off in self-supervised denoising. If the noise distribution is fully known, SURE provides the most expressive estimator, matching the performance of supervised learning. As the assumptions on the noise are relaxed, the learned estimator needs to be less expressive to avoid over-fitting the noise. In this work, we show that popular 'Noise2x' strategies impose too restrictive conditions on the learned denoiser, and propose an alternative that strikes a better trade-off.
• Figure 2: Removing the expected divergence of pretrained MMSE denoisers. Considering the DRUNet as an approximate MMSE denoiser, we use the formula in \ref{['eq: divfree from mmse']} to evaluate its ZED version and plot the expected theoretical error according to \ref{['eq: theoretical DMSE1']}. The ZED correction only results in a loss of less than 1 dB for most noise levels. Moreover, the theoretical error follows very closely the empirical one.
• Figure 3: MNIST denoising experiments. From left to right: i) evolution of the Lagrange multiplier $\eta$ for the case with $\sigma=0.2$, ii) test PSNR during training for UNSURE and UNSURE via score with $\sigma=0.1$, iii) estimated Lagrange multiplier as a function of $\sigma$, iv) average test PSNR for various methods and various noise levels $\sigma$.
• Figure 4: Image reconstruction results for various imaging problems. Top: colored Gaussian noise on DIV2K. Middle: Accelerated magnetic resonance imaging with FastMRI. Bottom: computed tomography with Poisson-Gaussian noise on LIDC
• Figure 5: Blind denoising results of Cryo-EM data using the PG-UNSURE method and Noise2Noise bepler_topaz-denoise_2020. Since the image is very large $7676 \times 7420$ pixels, is recommended to zoom in to observe the details.

Theorems & Definitions (6)

• Proposition 1
• Proposition 2
• Theorem 3
• proof
• proof
• Lemma 4