Denoising: A Powerful Building-Block for Imaging, Inverse Problems, and Machine Learning

TL;DR

The paper reframes denoising as a foundational building block rather than a mere noise-reduction step, formalizing ideal denoisers with an identity at zero noise and a conservative, energy-based structure. It shows how denoisers enable natural multiscale image decompositions, connect to score-based models via Tweedie’s formula, and serve as priors in inverse problems through RED and plug-and-play frameworks. The authors categorize denoisers into Bayesian (MAP/MMSE), energy-based, and kernel families, and highlight how each yields a gradient-of-energy interpretation and practical links to diffusion models and posterior sampling. Overall, denoising methods unify Bayesian estimation, energy-based modeling, and generative dynamics, enabling powerful reconstruction, principled regularization, and scalable generative capabilities across imaging, inverse problems, and machine learning. Key mathematical relationships include $\hat{\mathbf{x}}_{mmse} = \mathbf{x} + \alpha \nabla \log P(\mathbf{x},\alpha)$ and $f(\mathbf{x},\alpha) = \nabla \mathcal{E}(\mathbf{x},\alpha)$, which underpin score-based modeling and energy-based interpretations, as well as the diffusion-flow formulation $\frac{d\mathbf{x}_t}{dt} = -\tfrac{1}{2}\frac{d\alpha_t}{dt}\nabla \log P(\mathbf{x}_t,\alpha_t)$.

Abstract

Denoising, the process of reducing random fluctuations in a signal to emphasize essential patterns, has been a fundamental problem of interest since the dawn of modern scientific inquiry. Recent denoising techniques, particularly in imaging, have achieved remarkable success, nearing theoretical limits by some measures. Yet, despite tens of thousands of research papers, the wide-ranging applications of denoising beyond noise removal have not been fully recognized. This is partly due to the vast and diverse literature, making a clear overview challenging. This paper aims to address this gap. We present a clarifying perspective on denoisers, their structure, and desired properties. We emphasize the increasing importance of denoising and showcase its evolution into an essential building block for complex tasks in imaging, inverse problems, and machine learning. Despite its long history, the community continues to uncover unexpected and groundbreaking uses for denoising, further solidifying its place as a cornerstone of scientific and engineering practice.

Figures (6)

• Figure 1: Denoising as a natural image decomposition. Image adapted from talebi2016fast.
• Figure 2: Bayesian Denoisers: MAP vs. MMSE.
• Figure 3: Example of MAP denoiser with $L_1$ loss, with $\alpha = 1$. The Moreau envelope is the Huber loss.
• Figure 4: Illustration of a one-dimensional MMSE denoiser employing $L_1$ regularization, demonstrating the impact of varying $\alpha$. The visualization progresses from the smoothed distribution $P(\mathbf{x}, \alpha)$ (left), to the corresponding Energy function (middle), and ultimately, the resulting denoiser (right).
• Figure 5: Left: Example of MMSE denoiser with $L_1$ loss, with $\alpha = 1$. Right: Comparison of MAP and MMSE denoiser for the $L1$ loss, with $\alpha=1$.