Image denoising as a conditional expectation

TL;DR

This paper reframes image denoising as estimating the true noise-free image via a conditional expectation with respect to an underlying joint measure, avoiding explicit prior noise models. It embeds the problem into reproducing kernel Hilbert space theory, using kernel integral operators to convert the conditional expectation into a data-driven regression problem that can be solved with finite-dimensional matrices. Convergence is established: as the number of pixels grows and the regularization vanishes, the estimator uniformly converges to the true image, with a patch-based, scalable implementation and an explicit bandwidth-tuning strategy. The approach unifies local averaging, kernel regression, and RKHS methods, and demonstrates effectiveness on synthetic and real images, while acknowledging boundary artifacts and proposing avenues for enhancement with total variation or diffusion-based refinements.

Abstract

All techniques for denoising involve a notion of a true (noise-free) image, and a hypothesis space. The hypothesis space may reconstruct the image directly as a grayscale valued function, or indirectly by its Fourier or wavelet spectrum. Most common techniques estimate the true image as a projection to some subspace. We propose an interpretation of a noisy image as a collection of samples drawn from a certain probability space. Within this interpretation, projection based approaches are not guaranteed to be unbiased and convergent. We present a data-driven denoising method in which the true image is recovered as a conditional expectation. Although the probability space is unknown apriori, integrals on this space can be estimated by kernel integral operators. The true image is reformulated as the least squares solution to a linear equation in a reproducing kernel Hilbert space (RKHS), and involving various kernel integral operators as linear transforms. Assuming the true image to be a continuous function on a compact planar domain, the technique is shown to be convergent as the number of pixels goes to infinity. We also show that for a picture with finite number of pixels, the convergence result can be used to choose the various parameters for an optimum denoising result.

Figures (8)

• Figure 1: Kernel based denoising - an example. The article presents a data-driven technique for denoising which does not assume any prior distribution for the noise. The denoised image is interpreted as a conditional expectation. The numerical method is thus a means of estimating the conditional expectation from samples distributed uniformly with respect to an underlying measure. The panels show the results of applying this technique to a real world image. The original image has several separate layers, such as a foreground, a background, and other objects in between. Each of the layers have different textures. The denoised image and its errors from the noise-free image show the effectiveness of this approach to denoising.
• Figure 2: Kernel based denoising - Experiment 4. The goal is the same as in Figure \ref{['fig:shundorbon2']}. Here the image has a more continuous background and with a non-smooth, low dimensional object in the middle. The RKHS based technique shows the most error at these points of non-smoothness.
• Figure 3: Kernel based denoising - Experiment 5. The goal is the same as in Figures \ref{['fig:shundorbon2']} and \ref{['fig:howrah_bridge']}. Here the image is more smooth, but has several objects with fractal outlines. These outlines are reproduced in the heatmap plot of the error.
• Figure 4: Mathematical principle for denoising. The algorithmic steps (white) as well as the theoretical rationale (grey) are outlined in this diagram. Our approach is based on an interpretation of the joint color and pixel location as a sampling of a probability space. That way we convert the denoising problem into the task of finding a conditional expectation. In the above diagram, $\bar{f}$ and $f$ are functional representations of the true and noisy images respectively. See \ref{['eqn:def:img_noise']} and \ref{['eqn:scheme:1']} for more details. The kernel based approach for estimating $\bar{f}$ from pixel information is presented in Section \ref{['sec:technique']}. The steps outlined here pertain to the last algorithmic step in the flowchart of Figure \ref{['fig:outline:2']}.
• Figure 5: Outline of the algorithms. All the algorithmic steps involved in the denoising algorithm are presented, along with the intermediate objects. The input and output objects are shown in green, algorithmic parameters in blue, and intermediate objects as yellow. The steps outlined in Figure \ref{['fig:outline:1']} pertain to the last algorithmic step in this flowchart. It is also described in detail in Algorithm \ref{['algo:1']}. The auto-tuning process is described in Algorithm \ref{['algo:2']}. The flowchart reveals that the kernel denoiser labelled as KDn is based only on the grid size, and is independent of the color information. KDn combines linearly with the color information in the final step. The details of that step is also depicted separately in Figure \ref{['fig:linear_filter']}.

Theorems & Definitions (3)

• Lemma 2.1
• Theorem 1
• Corollary 2