The Little Engine that Could: Regularization by Denoising (RED)

TL;DR

This work introduces Regularization by Denoising (RED), a general framework that leverages any image denoising engine f as an explicit, image-adaptive regularizer for inverse problems. By defining a Laplacian-like prior ρ_{L}(\mathbf{x}) = \frac{1}{2} \mathbf{x}^T(\mathbf{x} - f(\mathbf{x})) and energy E(\mathbf{x}) = ℓ(\mathbf{y},\mathbf{x}) + \frac{\lambda}{2} \mathbf{x}^T(\mathbf{x} - f(\mathbf{x})), RED yields a tractable gradient ∇_{\mathbf{x}} E(\mathbf{x}) = ∇_{\mathbf{y}} ℓ(\mathbf{y},\mathbf{x}) + \lambda(\mathbf{x} - f(\mathbf{x})). The authors show convexity of ρ_{L} under mild conditions on f, provide multiple optimization schemes (gradient descent, ADMM, fixed-point) to minimize E, and demonstrate state-of-the-art results in image deblurring and single-image super-resolution while highlighting RED's robustness and simpler parameter tuning relative to the Plug-and-Play Prior (PPP). The work also analyzes the relationship to PPP, showing equivalence only in highly restrictive scenarios, and discusses extensions and unresolved questions such as the choice of the denoiser noise level σ_f and applicability to non-differentiable denoisers.

Abstract

Removal of noise from an image is an extensively studied problem in image processing. Indeed, the recent advent of sophisticated and highly effective denoising algorithms lead some to believe that existing methods are touching the ceiling in terms of noise removal performance. Can we leverage this impressive achievement to treat other tasks in image processing? Recent work has answered this question positively, in the form of the Plug-and-Play Prior ($P^3$) method, showing that any inverse problem can be handled by sequentially applying image denoising steps. This relies heavily on the ADMM optimization technique in order to obtain this chained denoising interpretation. Is this the only way in which tasks in image processing can exploit the image denoising engine? In this paper we provide an alternative, more powerful and more flexible framework for achieving the same goal. As opposed to the $P^3$ method, we offer Regularization by Denoising (RED): using the denoising engine in defining the regularization of the inverse problem. We propose an explicit image-adaptive Laplacian-based regularization functional, making the overall objective functional clearer and better defined. With a complete flexibility to choose the iterative optimization procedure for minimizing the above functional, RED is capable of incorporating any image denoising algorithm, treat general inverse problems very effectively, and is guaranteed to converge to the globally optimal result. We test this approach and demonstrate state-of-the-art results in the image deblurring and super-resolution problems.

Figures (8)

• Figure 1: An empirical evaluation of the homogeneity property. These graphs show $f((1+\epsilon)\textbf{x})$ versus $(1+\epsilon)f(\textbf{x})$ as a scatter-plot for K-SVD, BM3D, NLM, EPLL, and the TNRD. Equality implies satisfaction of the homogeneity, and the numbers in the brackets provide the STD of the difference. Note that these results were observed on various test images, but shown here for the image Peppers.
• Figure 4: The Proposed Scheme (RED) via the fixed-point method.
• Figure 5: An illustration of the convergence of RED using three proposed numerical schemes -- the steepest-descent (red), the ADMM with $m_2 = 1$ (green), the ADMM with $m_2 = 3$ (blue) and the fixed-point (black). These are applied on the image Leaves, degraded by a Gaussian PSF, when two denoising engines are tested: (a) the median filter, and (b) TNRD chen2015trainable.
• Figure 6: Visual comparison of deblurring a cropped area from the image Starfish, degraded by a uniform PSF, along with the corresponding PSNR [dB] score.
• Figure 7: Visual comparison of deblurring a cropped area from the image Leaves, degraded by a Gaussian PSF, along with the corresponding PSNR [dB] score.