Zero-shot Denoising via Neural Compression: Theoretical and algorithmic framework

TL;DR

This work tackles zero-shot denoising in settings where clean targets are unavailable by training a neural compression model directly on patches from a single noisy image. The proposed ZS-NCD framework leverages an entropy-regularized, patch-wise neural compression objective, followed by averaging overlapping patch reconstructions to form the final denoised image. The authors provide non-asymptotic, finite-sample guarantees for compression-based denoisers under Gaussian and Poisson noise, including specializations to sparse signals and Poisson settings. Empirically, ZS-NCD achieves state-of-the-art performance among zero-shot denoisers across natural and non-natural domains, including fluorescence microscopy and real camera data, and demonstrates robustness to overfitting through entropy constraints and patch-aggregation effects. The work thus advances both the theoretical foundations and practical efficacy of compression-based denoising in a truly self-contained zero-shot framework, with publicly available code for replication.

Abstract

Zero-shot denoising aims to denoise observations without access to training samples or clean reference images. This setting is particularly relevant in practical imaging scenarios involving specialized domains such as medical imaging or biology. In this work, we propose the Zero-Shot Neural Compression Denoiser (ZS-NCD), a novel denoising framework based on neural compression. ZS-NCD treats a neural compression network as an untrained model, optimized directly on patches extracted from a single noisy image. The final reconstruction is then obtained by aggregating the outputs of the trained model over overlapping patches. Thanks to the built-in entropy constraints of compression architectures, our method naturally avoids overfitting and does not require manual regularization or early stopping. Through extensive experiments, we show that ZS-NCD achieves state-of-the-art performance among zero-shot denoisers for both Gaussian and Poisson noise, and generalizes well to both natural and non-natural images. Additionally, we provide new finite-sample theoretical results that characterize upper bounds on the achievable reconstruction error of general maximum-likelihood compression-based denoisers. These results further establish the theoretical foundations of compression-based denoising. Our code is available at: https://github.com/Computational-Imaging-RU/ZS-NCDenoiser.

Figures (9)

• Figure 1: Zero-Shot Neural Compression Denoiser (ZS-NCD). Learning phase: a neural compression model (architecture shown in Fig. \ref{['fig:neural-net']} of the supplementary material) is trained on overlapping patches extracted from a single noisy image. Denoising phase: each pixel is reconstructed by averaging predictions across neighboring patches processed by the trained model.
• Figure 2: Zero-shot denoising of Kodim05 with AWGN ($\sigma=25$). Left: PSNR versus training iterations for zero-shot denoisers. Performance of BM3D dabov2007bm3d and Restormer zamir2022restormer are included as a classical baseline and as a supervised empirical upper bound, respectively. Right: Visual reconstructions with PSNR above each image. Compression-based denoising based on JPEG-2K taubman2002jpeg2000 achieves inferior performance. Learning-based zero-shot denoisers often struggle with either overfitting or high bias. DIP dmitry2020dip and DD heckel2018deep require early stopping to avoid overfitting. ZS-N2S batson2019noise2self and S2S quan2020self2self struggle with high-resolution color images, and ZS-N2N mansour2023zsn2n often produces noisy outputs with potential overfitting. BM3D tends to oversmooth the denoised image. In contrast, ZS-NCD avoids these issues.
• Figure 3: Finding Lagrangian coefficient $\lambda$
• Figure 4: Denoising Parrot using ZS-NCD, where only a single pixel from each overlapping patch (stride 1) is retained after compression. (AWGN, $\sigma_z = 25$.) Each heatmap value indicates the PSNR achieved when denoising is based solely on the pixel at that specific location within each patch.
• Figure 5: Neural compression network used for denoising. Conv and FC denote the convolutiona and fully connected layer, respectively. GDN and ReLU are activation functions.

Theorems & Definitions (13)

• Theorem 1
• Corollary 1: AWGN, sparse signals
• Theorem 2
• Theorem 3
• Remark 1
• Lemma 1
• Lemma 2
• proof
• proof
• proof