Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Linear Combinations of Patches are Unreasonably Effective for Single-Image Denoising

Sébastien Herbreteau, Charles Kervrann

TL;DR

This paper tackles single-image denoising without external training data, addressing the weaknesses of supervised methods that rely on high-quality datasets. It introduces LIChI, a parametric, patch-based denoiser that uses linear and iterative combinations of similar patches arranged in a chaining framework to progressively refine estimates. The approach leverages multiple pilots and risk-based criteria (including SURE and Noisier2Noise) with an aggregation step to fuse a large set of patch-based estimates, achieving competitive results. Experiments on Gaussian and real-world noise show LIChI matches or surpasses the best self-supervised methods while remaining fast and fully interpretable, highlighting the potential of linear patch combinations for zero-shot denoising.

Abstract

In the past decade, deep neural networks have revolutionized image denoising in achieving significant accuracy improvements by learning on datasets composed of noisy/clean image pairs. However, this strategy is extremely dependent on training data quality, which is a well-established weakness. To alleviate the requirement to learn image priors externally, single-image (a.k.a., self-supervised or zero-shot) methods perform denoising solely based on the analysis of the input noisy image without external dictionary or training dataset. This work investigates the effectiveness of linear combinations of patches for denoising under this constraint. Although conceptually very simple, we show that linear combinations of patches are enough to achieve state-of-the-art performance. The proposed parametric approach relies on quadratic risk approximation via multiple pilot images to guide the estimation of the combination weights. Experiments on images corrupted artificially with Gaussian noise as well as on real-world noisy images demonstrate that our method is on par with the very best single-image denoisers, outperforming the recent neural network based techniques, while being much faster and fully interpretable.

Linear Combinations of Patches are Unreasonably Effective for Single-Image Denoising

TL;DR

This paper tackles single-image denoising without external training data, addressing the weaknesses of supervised methods that rely on high-quality datasets. It introduces LIChI, a parametric, patch-based denoiser that uses linear and iterative combinations of similar patches arranged in a chaining framework to progressively refine estimates. The approach leverages multiple pilots and risk-based criteria (including SURE and Noisier2Noise) with an aggregation step to fuse a large set of patch-based estimates, achieving competitive results. Experiments on Gaussian and real-world noise show LIChI matches or surpasses the best self-supervised methods while remaining fast and fully interpretable, highlighting the potential of linear patch combinations for zero-shot denoising.

Abstract

In the past decade, deep neural networks have revolutionized image denoising in achieving significant accuracy improvements by learning on datasets composed of noisy/clean image pairs. However, this strategy is extremely dependent on training data quality, which is a well-established weakness. To alleviate the requirement to learn image priors externally, single-image (a.k.a., self-supervised or zero-shot) methods perform denoising solely based on the analysis of the input noisy image without external dictionary or training dataset. This work investigates the effectiveness of linear combinations of patches for denoising under this constraint. Although conceptually very simple, we show that linear combinations of patches are enough to achieve state-of-the-art performance. The proposed parametric approach relies on quadratic risk approximation via multiple pilot images to guide the estimation of the combination weights. Experiments on images corrupted artificially with Gaussian noise as well as on real-world noisy images demonstrate that our method is on par with the very best single-image denoisers, outperforming the recent neural network based techniques, while being much faster and fully interpretable.
Paper Structure (23 sections, 3 theorems, 42 equations, 9 figures, 3 tables, 2 algorithms)

This paper contains 23 sections, 3 theorems, 42 equations, 9 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. A parametric view of two-step non-local methods for single-image denoising
  3. A unified framework for non-local denoisers
  4. Parameter optimization
  5. Principle of internal adaptation
  6. LIChI: linear and iterative combinations of patches for image denoising
  7. A novel chaining rule for generalization
  8. A progressive scheme for parameter optimization
  9. Resolution when the true image is available
  10. Use of multiple cost-efficient pilots to guide estimation
  11. Weighted average reprojection
  12. Building an initial pilot
  13. Stein's unbiased risk estimate (SURE)
  14. Noisier2Noise
  15. Two additional extreme pilots
  16. ...and 8 more sections

Key Result

Lemma 1

Let $A \in \mathbb{R}^{n \times k}$, $\lambda \in \mathbb{R}^{+}$ and $\mu \in \mathbb{R}$. If $A^\top A$ is invertible or $\lambda \neq 0$:

Figures (9)

  • Figure 1: The execution time on CPU for an image of size $512\times512$ v.s the average PSNR results on Set12 and BSD68 berkeley for synthetic Gaussian noise with $\sigma=25$ of the most effective popular methods drunetdncnnffdnetLIDIAscunetrestormernlridgenlbayesBM3DWNNMEPLL_unsupervisedTWSCrethinkingS2S. These results are calculated based on Table \ref{['resultsPSNR']} and subsection \ref{['section_complexity']}.
  • Figure 2: Illustration of the parametric view of several popular non-local denoisers BM3Dnlbayesnlridge. Examples of parameterized functions $f_{\Theta_i}$, unequivocally identifying the denoiser, are given whose parameters $\Theta_i$ are eventually selected for each group of patches by "internal adaptation" (see equation \ref{['risklocal1emp']}).
  • Figure 3: Evolution of the PSNR for pilot-based single-image denoising methods BM3Dnlbayesnlridge on Set12 dataset with noise level $\sigma=25$. After a PSNR jump between the first and second pilot, obtained with "internal adaptation", the PSNR decreases for all methods, except ours.
  • Figure 4: Average PSNR (in dB) results on patch groups (dotted line), after aggregation (dashed line) and when taken as input for Algorithm \ref{['algo1']} (solid line) for Set12 dataset depending on combination weights used and noise level. Patch and group sizes are chosen as indicated by Table \ref{['optimalParams']}.
  • Figure 5: Colormap of the PSNR (in dB) of the denoised blocks of similar patches ($n=7 \times 7$ and $k = 18$) associated with each overlapping patch of the noisy image. The average PSNR on blocks of similar patches is also indicated.
  • ...and 4 more figures

Theorems & Definitions (6)

  • Lemma 1
  • proof
  • Proposition 1
  • proof
  • Proposition 2: Noisier2Noise
  • proof