Stimulating Diffusion Model for Image Denoising via Adaptive Embedding and Ensembling

TL;DR

This work presents a novel strategy called the Diffusion Model for Image Denoising (DMID) by understanding and rethinking the diffusion model from a denoising perspective, which achieves state-of-the-art performance on both distortion-based and perception-based metrics, for both Gaussian and real-world image denoising.

Abstract

Image denoising is a fundamental problem in computational photography, where achieving high perception with low distortion is highly demanding. Current methods either struggle with perceptual quality or suffer from significant distortion. Recently, the emerging diffusion model has achieved state-of-the-art performance in various tasks and demonstrates great potential for image denoising. However, stimulating diffusion models for image denoising is not straightforward and requires solving several critical problems. For one thing, the input inconsistency hinders the connection between diffusion models and image denoising. For another, the content inconsistency between the generated image and the desired denoised image introduces distortion. To tackle these problems, we present a novel strategy called the Diffusion Model for Image Denoising (DMID) by understanding and rethinking the diffusion model from a denoising perspective. Our DMID strategy includes an adaptive embedding method that embeds the noisy image into a pre-trained unconditional diffusion model and an adaptive ensembling method that reduces distortion in the denoised image. Our DMID strategy achieves state-of-the-art performance on both distortion-based and perception-based metrics, for both Gaussian and real-world image denoising.The code is available at https://github.com/Li-Tong-621/DMID.

Figures (17)

• Figure 1: From the denoising perspective, the reverse process of a diffusion model can be viewed as a coarse-to-fine iterative denoising process. The noise level corresponds to the standard deviation of noise on an 8-bit image, where the maximum signal for a clean image is 255.
• Figure 2: Perception-distortion trade-off of different methods. Our method traverses through the perception-distortion curve and achieves SOTA performance.
• Figure 3: Our Diffusion Model for Image Denoising (DMID) strategy. Our adaptive embedding method connects diffusion model and image denoising, while our adaptive ensemble method reduces distortion in the final denoised image.
• Figure 4: The procedure for the embedding method. The embedding method first transforms the noise into Gaussian noise and subsequently normalizes the latent noisy image. After that, we search all the timestep $t$ to find a timestep $N$ to guarantee $\frac{\sqrt{1-\bar{\alpha}_{N}}}{\sqrt{\bar{\alpha}_{N}}}$ closest to $\sigma$. Ultimately, we multiply the noisy image by $\sqrt{\bar{\alpha}_N}$ to convert the image to the intermediate state $x_N$.
• Figure 5: Visual results on classical Gaussian image denoising. The images restored by our model exhibit more details and realism.