Beyond sparse denoising in frames: minimax estimation with a scattering transform

TL;DR

The paper tackles denoising of images corrupted by Gaussian noise by moving beyond fixed-frame sparsity to adaptively capture geometric regularity. It introduces a denoising approach based on the wavelet scattering transform, where a joint minimisation/maximisation of scattering-$\ell^1$ norms encodes both directional regularity along edges and the sharp edge profile, connecting harmonic analysis with deep-learning-inspired representations. Numerical results indicate that this scattering-based denoiser can attain minimax rates for ${\bf C}^\alpha$ edges with $\alpha\le 2$, and a mathematical conjecture links these empirical findings to a rigorous minimax bound up to a $|\log \sigma|$ factor. The work thus provides a mathematical bridge between traditional harmonic-analysis denoising, adaptive geometric models, and the practical performance of deep neural network estimators, with potential implications for designing robust, geometry-aware denoisers.

Abstract

A considerable amount of research in harmonic analysis has been devoted to non-linear estimators of signals contaminated by additive Gaussian noise. They are implemented by thresholding coefficients in a frame, which provide a sparse signal representation, or by minimising their $\ell^1$ norm. However, sparse estimators in frames are not sufficiently rich to adapt to complex signal regularities. For cartoon images whose edges are piecewise $\bf C^α$ curves, wavelet, curvelet and Xlet frames are suboptimal if the Lipschitz exponent $α\leq 2$ is an unknown parameter. Deep convolutional neural networks have recently obtained much better numerical results, which reach the minimax asymptotic bounds for all $α$. Wavelet scattering coefficients have been introduced as simplified convolutional neural network models. They are computed by transforming the modulus of wavelet coefficients with a second wavelet transform. We introduce a denoising estimator by jointly minimising and maximising the $\ell^1$ norms of different subsets of scattering coefficients. We prove that these $\ell^1$ norms capture different types of geometric image regularity. Numerical experiments show that this denoising estimator reaches the minimax asymptotic bound for cartoon images for all Lipschitz exponents $α\leq 2$. We state this numerical result as a mathematical conjecture. It provides a different harmonic analysis approach to suppress noise from signals, and to specify the geometric regularity of functions. It also opens a mathematical bridge between harmonic analysis and denoising estimators with deep convolutional network.

Figures (14)

• Figure 1: (a) A $\mathrm C^2$-geometrically regular image discretised in dimension $d=128^2$, (b) 3 levels discrete orthogonal wavelet decomposition of this image. Large wavelet coefficients could clearly be seen along edges. Symlets with four vanishing moments are used Daubechies1992TenLO.
• Figure 2: (a): the noisy image is the sum of a $\bf C^2$ geometrically regular image ranging in $[-1,1]$ with an additive Gaussian white noise of variance $\sigma^2 = 0.005$. (b): orthogonal wavelet coefficients of the noisy image computed on $3$ scales $2^j$. Larger coefficients are darker. (c): non-zero wavelet coefficients after soft thresholding are shown in black. (d): denoised estimated image recovered from thresholded wavelet coefficients. The PSNR is 28.97dB for the noisy image and 33.29dB for its denoised counterpart.
• Figure 3: (a): Mean-squared error $\epsilon_{ms} (\sigma)$ of a wavelet thresholding estimator, as a function of the noise variance $\sigma^2$, computed on average over $\mathbf C^{\alpha}$ geometrically regular images defined in \ref{['appendix:discretising_geometry']}. Red to blue curves correspond to Lipschitz exponent $\alpha \in \{2, 1.5, 1.2, 1\}$. The slope of $\log \epsilon_{ms}$ remains equal to $1$ as a function of $\log \sigma$. (b): same error calculations for a convolutional neural network estimator computed with a UNet. For each Lipschitz exponent $\alpha$, $\log \epsilon_{ms}$ nearly superimposes with a straight line of slope $2 \alpha / (\alpha + 1)$ which corresponds to the minimax rate.
• Figure 4: (a) An edge creates large wavelet coefficients whenever the square wavelet supports intersect the edge and the wavelets oscillate across the edge. The support width is proportional to the scale $2^j$ so the total number of large wavelet coefficients along an edge is proportional to $2^{-j}$. (b) Curvelets, shearlets and bandlets have elongated supports whose width is proportional to $2^j$ but whose length is proportional to $2^{j/\alpha}$, with $\alpha = 2$ for curvelets and shearlets. If the edge is a ${\bf C}^{\alpha}$ curve then the number of non-negligible coefficients is proportional to $2^{-j/\alpha}.$
• Figure 5: This figure shows the denoising performance of a UNet trained on two datasets. For the top row image, the UNet was trained on the LSUN bedrooms dataset yu15lsun in dimension $d=128^2$ pixels. On the second row it was trained on $\mathbf C^{\alpha}$ geometrically regular images of same size. The image amplitudes are normalized in $[-1,1]$. The first column shows an original test image which does not belong to the training set. The second column shows a noisy image with a noise variance $\sigma^2=1$, which corresponds to a PSNR of $6$dB. The third column displays the denoised image computed by the UNet. The PSNR is $22.03$ dB for the bedroom image $30.03$dB for the $\mathbf C^{\alpha}$ geometrically regular image. The fourth column shows a zoom near edges. In both case the estimator reproduces very well the image edges and corners.

Theorems & Definitions (12)

• Definition 2.1
• Theorem 3.1
• Theorem 3.2
• Conjecture 3.3
• Lemma A.1
• proof
• Theorem
• proof
• Lemma C.1
• proof