Denoising Diffusion Post-Processing for Low-Light Image Enhancement

TL;DR

This work proposes using a diffusion model as a post-processing approach, and introduces Low-light Post-processing Diffusion Model (LPDM) in order to model the conditional distribution between under-exposed and normally-exp exposed images.

Abstract

Low-light image enhancement (LLIE) techniques attempt to increase the visibility of images captured in low-light scenarios. However, as a result of enhancement, a variety of image degradations such as noise and color bias are revealed. Furthermore, each particular LLIE approach may introduce a different form of flaw within its enhanced results. To combat these image degradations, post-processing denoisers have widely been used, which often yield oversmoothed results lacking detail. We propose using a diffusion model as a post-processing approach, and we introduce Low-light Post-processing Diffusion Model (LPDM) in order to model the conditional distribution between under-exposed and normally-exposed images. We apply LPDM in a manner which avoids the computationally expensive generative reverse process of typical diffusion models, and post-process images in one pass through LPDM. Extensive experiments demonstrate that our approach outperforms competing post-processing denoisers by increasing the perceptual quality of enhanced low-light images on a variety of challenging low-light datasets. Source code is available at https://github.com/savvaki/LPDM.

Figures (7)

• Figure 1: Qualitative results of our proposed approach on a variety of evaluation datasets for a variety of LLIE methods. Left: the input test image. Middle: the output of the listed LLIE method for the input image. Right: the result of applying our proposed post-processing LPDM to the image from the middle column. The LPDM parameters used are $\phi = 300$ and $s = 30$.
• Figure 2: Residual block used throughout the DM architecture consisting of a combination of group normalization layers, SiLU activations, convolution layers and addition operations. Both $c_{in}$ and $t$ are inputs to the residual block and represent the channel and timestep-embedded input respectively. Note that $t$ is already in embedded form when it enters the residual block. The output of the residual block is represented by $c_{out}$.
• Figure 3: Visualization of different components within the diffusion process and the LPDM pipeline. The first row displays a normally-exposed image, under-exposed image and image which has undergone LLIE. The second row demonstrates how noise is added to $\bm{x}_0$ using a linear variance schedule during the training process, with $T = 1000$. The third row demonstrates the effect of different values of $\phi$ in \ref{['eq:proposed:noise-estimate']}. The fourth row demonstrates the effect of applying \ref{['eq:diffusion:predict-x0-mod']} with different values of $s$ in order to enhance $\bm{\hat{x}}_0^\eta$.
• Figure 4: Diagram presenting the training phase and inference phase of the LPDM, displayed on the left and right half of the diagram respectively.
• Figure 5: A qualitative comparison of the BM3D ref:bm3d and NAFNet ref:nafnet-denoiser post-processing denoising approaches to LPDM on the LOL test set. The first column displays $\bm{\hat{x}}_0^\eta$ for different $\eta$. The final column contains the ground truth label. The remaining columns display the results of different denoising approaches which post-process $\bm{\hat{x}}_0^\eta$.