Fast, nonlocal and neural: a lightweight high quality solution to image denoising
Yu Guo, Axel Davy, Gabriele Facciolo, Jean-Michel Morel, Qiyu Jin
TL;DR
Problem: CNN-based denoisers deliver high PSNR but are computationally expensive, limiting mobile deployment, while nonlocal methods preserve texture but lag in PSNR. Approach: a hybrid pipeline combines a GPU-accelerated nonlocal denoising stage (BM3D-G) with a lightweight residual CNN and a flexible, noise-level conditioned U-Net denoiser trained with an $L_1$ loss $\mathcal{L}(\Theta)$. The loss is defined as $\mathcal{L}(\Theta)=\frac{1}{N}\sum_{i=1}^N|x_i-[F(z_i;\Theta)+BM3D(y_i)]|$, with $z_i=\text{Concatenate}(BM3D(y_i),y_i)$. Contributions: (i) integration of BM3D preprocessing with a shallow CNN to reduce model size while maintaining quality; (ii) a 4-encoder/4-decoder U-Net using depthwise separable convolutions and a noise map for varying noise levels; (iii) empirical results showing between $10$ and $20$ times speed-up over CNNs with equivalent performance and higher PSNR, notably on MIT Moiré textures. Significance: enables high-quality denoising on mobile devices with modest computational requirements and can accommodate alternative fast nonlocal filters.
Abstract
With the widespread application of convolutional neural networks (CNNs), the traditional model based denoising algorithms are now outperformed. However, CNNs face two problems. First, they are computationally demanding, which makes their deployment especially difficult for mobile terminals. Second, experimental evidence shows that CNNs often over-smooth regular textures present in images, in contrast to traditional non-local models. In this letter, we propose a solution to both issues by combining a nonlocal algorithm with a lightweight residual CNN. This solution gives full latitude to the advantages of both models. We apply this framework to two GPU implementations of classic nonlocal algorithms (NLM and BM3D) and observe a substantial gain in both cases, performing better than the state-of-the-art with low computational requirements. Our solution is between 10 and 20 times faster than CNNs with equivalent performance and attains higher PSNR. In addition the final method shows a notable gain on images containing complex textures like the ones of the MIT Moire dataset.