Image Denoising Using the Geodesics' Gramian of the Manifold Underlying Patch-Space

TL;DR

This work introduces Geodesic Gramian Denoising (GGD), a patch-based, non-local image denoising method that operates in the patch-space manifold. By partitioning an image into overlapping patches, building a graph to approximate geodesic distances, and transforming the resulting geodesic distance matrix into a Gramian, GGD uses the leading eigenvectors to reconstruct denoised patches, which are then merged with Shepard's method. Sensitivity analyses show how patch size, neighborhood size, corruption level, and eigenvector threshold influence performance, with GGD often outperforming BM3D, KSVD, BWD, NLB, AD, and ID in preserving texture and edges. Although computationally intensive due to the $n^2\times n^2$ Gramian and $O(n^6)$ eigen-decomposition, the authors propose future work employing Lanczos, randomized, and Monte Carlo techniques to enable real-time applicability.

Abstract

With the proliferation of sophisticated cameras in modern society, the demand for accurate and visually pleasing images is increasing. However, the quality of an image captured by a camera may be degraded by noise. Thus, some processing of images is required to filter out the noise without losing vital image features. Even though the current literature offers a variety of denoising methods, the fidelity and efficacy of their denoising are sometimes uncertain. Thus, here we propose a novel and computationally efficient image denoising method that is capable of producing accurate images. To preserve image smoothness, this method inputs patches partitioned from the image rather than pixels. Then, it performs denoising on the manifold underlying the patch-space rather than that in the image domain to better preserve the features across the whole image. We validate the performance of this method against benchmark image processing methods.

Figures (5)

• Figure 1: Information retention of eigenvalues and eigenvectors of the Gramian matrix constructed for an image of size 100$\times$100. First, we construct the patch-set of $100^2$ patches, each with the length $\rho=5$, from the leftmost image in (a). Second, we create a graph structure in the patch-space with the neighborhood size $\delta=10$ and approximate the geodesic distances using Floyd's algorithm. Third, we formulate the Gramian matrix of the geodesic distances. We reconstruct the leftmost image in (a) using the eigenvector associated with the biggest eigenvalue, see the second image from the left of (a), using the eigenvectors associated with the five biggest eigenvalues, see the third image from the left of (a), and using the eigenvectors associate with the 100 biggest eigenvalues, see the rightmost image in (a). (b) The magnitude of eigenvalues, denoted by $\lambda_l$ for $l=1,\dots,n^2$, from the biggest to the smallest, versus eigenvalue index, denoted by $l$, of the Gramian matrix and, (c) cumulative percentage of eigenvalues, denoted by $\tilde{\lambda}_l$ for $l=1,\dots,n^2$, versus the index.
• Figure 2: Denoising performance of GGD with respect to patch size ($\rho$) and neighborhood size ($\delta$). We produce three noisy images from an image of Lena Forsén of size $100\times 100$ by imposing three levels of noise sampled from the Gaussian distributions $\mathcal{N}(0,\epsilon^2)$ where $\epsilon=40$, 60, and 80. We set the eigenvector threshold to an arbitrary 50 and run GGD with 36 pairs of parameter values $\{(\rho,\delta)\vert \ \rho = 3,5,7,9,11,13; \delta = 5,10,15,20,25,30\}$ on each of the three images. Then, we compute the reconstruction errors in terms of RMSE, PSNR, and SSIM of GGD associated with 36 parameter pairs and three noise levels $\epsilon=40, 60, 80$. We run the same experiment four more times each with a different seed (each contributes a sample realization) in the Gaussian distribution that generates additive noise for the corrupted. For each metric, (a-c) RMSE, (d-f) PSNR, and (g-i) SSIM, and for each noise level $\epsilon = 40, 60$, and 80, we average the corresponding performance values over the five sample realizations. Each color-bar is scaled to interpret the values in all the three cases $\epsilon = 40, 60$, and 80 in the same row. We also compute the corruption level of the three noisy images using the same metric where $RMSE=38.76$, $PSNR=16.36$, and $SSIM=.3537$ for $\epsilon=40$; $RMSE=53.87$, $PSNR=13.50$, and $SSIM=.2444$ for $\epsilon=60$; $RMSE=66.82$, $PSNR=11.63$, and $SSIM=.1820$ for $\epsilon=80$. (j-l) The appearance of the noisy images and their denoised images for all the 3$\times$36 cases (see supplementary materials for the enlarged images).
• Figure 3: Image denoising performance of GGD with respect to the intensity of noise contamination ($\epsilon$). We produce 11 images by imposing a variable noise into an image of Lena Forsén of size $100\times 100$ that is sampled from the Gaussian distributions $\mathcal{N}(0,\epsilon^2)$ where $\epsilon=$ 0, 10, $\dots$, 100. We denoise these images using GGD with the parameters ($\rho, \delta, L$) for $\rho=3,7$; $\delta=10$; and $L=10, 10^2, 10^3, 10^4$ where $\delta$, $\rho$, and $L$ denote neighborhood size, patch size, and eigenvector threshold, respectively. We compute the reconstruction errors using the three metrics RMSE, PSNR, and SSIM for all the denoised versions of the noisy images. We run GGD with each of the parameter combinations nine more times with different random seeds (each random seed provides a sample realization) in the Gaussian distribution. For each parameter combination, the means of the reconstruction errors across 10 sample realizations with respect to $\epsilon$ is presented in (a-c) where errorbars represent the standard deviations of the reconstruction errors computed across 10 sample realizations. The corruption levels of the input images, denoted by $\mathcal{U}$, with respect to the noise levels is also computed using the three metrics. We use the corruption levels to compare the denoising results of GGD. (d) The appearance of the noisy images and their denoised images for all the 11 noisy images (see supplementary materials for the enlarged images).
• Figure 4: Denoising performance of GGD with respect to different eigenvector thresholds ($L$'s). We impose random noise sampled from the Gaussian distributions $\mathcal{N}(0,\epsilon^2)$, where $\epsilon=$ 40, 60, and 80, into the image "Lena Forsén" of size $100\times 100$, and generate three noisy test images. Each noisy image is denoised using GGD with four combinations of the parameters neighborhood size ($\delta$) and patch size ($\rho$) such that ($\delta$, $\rho$) = (10,5), (10,9), (20,5), and (20,9) for a sequence of eigenvector thresholds $L=50,100,\dots,10000$ (10000 is the total number of eigenvectors of an image of size $100\times 100$). The reconstruction error of the denoising is computed using three performance metrics RMSE, PSNR, and SSIM that we present in (a-i). Consider that K's in these plots stand for thousand (e.i, $'$000). The corruption levels of the three test images, denoted by $\mathcal{U}$'s, are also computed using the same three performance metrics and used them to compare the denoising performance of GGD. In (a-i), even though the amount of corruption computed using any of the performance metrics for any noisy images is both a scalar and independent of the eigenvector threshold parameter, we represent it as a horizontal line at the metric value across eigenvector thresholds to provide a better comparison. (j-l) The appearance of the noisy images and their denoised images for three noise levels, four parameter combinations, and four eigenvector thresholds (see supplementary materials for the enlarged images).
• Figure 5: Visual comparison of the quality of the images denoised by GGD with that of the six other benchmark denoising methods, namely, sparse 3-D transform-domain collaborative filtering (BM3D), sparse and redundant representations over learned dictionaries (KSVD), Bayes wavelet denoising (BWD), nonlocal Bayesian image denoising (NLB), anisotropic diffusion (AD), and isotropic diffusion (ID). Each of the five test images, namely, cameraman, mandrill, Barbara, boat, and clown, of size 150 $\times$ 150, is imposed with two Gaussian noise distributions with a mean of 0 and standard deviations ($\epsilon$'s) of 40 and 80. Here, the original noise-free images are denoted by $\mathcal{I}$'s and their noisy versions are denoted by $\mathcal{U}$'s (see supplementary materials for the enlarged images). While GGD is executed with the parameter values $\delta=20$, $\rho=7$, and $L=50$, the other methods are executed with their recommended or default parameter values. Here, we observe that GGD retains texture and cartoon in the denoised images most of the time than that of the other six methods.

Theorems & Definitions (6)

• Definition 1
• Definition 2
• Definition 3
• Definition 4
• Definition 5
• Definition 6