Advancing Unsupervised Low-light Image Enhancement: Noise Estimation, Illumination Interpolation, and Self-Regulation

TL;DR

This work tackles unsupervised low-light image enhancement under dynamic noise by introducing a fast gradient-based noise estimator, a Self-Calibrated Denoiser (SCD), and a Learnable Illumination Interpolator (LII) within a noise-aware, self-regularizing framework. The method interleaves denoising and illumination learning, using a reference-free loss that enforces natural color statistics and adaptive denoising constraints, achieving state-of-the-art performance on MIT and LOL datasets with PSNR improvements of $0.818$ dB and $0.675$ dB respectively, while running up to 100 frames per second. The approach emphasizes edge-preserving illumination, gradient-consistent illumination, and robust generalization to diverse lighting and noise conditions, with code and noise-estimation components available publicly. Overall, the proposed pipeline offers a practical, efficient solution for real-time LLIE that can enhance generalizability to other restoration tasks.

Abstract

Contemporary Low-Light Image Enhancement (LLIE) techniques have made notable advancements in preserving image details and enhancing contrast, achieving commendable results on specific datasets. Nevertheless, these approaches encounter persistent challenges in efficiently mitigating dynamic noise and accommodating diverse low-light scenarios. Insufficient constraints on complex pixel-wise mapping learning lead to overfitting to specific types of noise and artifacts associated with low-light conditions, reducing effectiveness in variable lighting scenarios. To this end, we first propose a method for estimating the noise level in low light images in a quick and accurate way. This facilitates precise denoising, prevents over-smoothing, and adapts to dynamic noise patterns. Subsequently, we devise a Learnable Illumination Interpolator (LII), which employs learnlable interpolation operations between the input and unit vector to satisfy general constraints between illumination and input. Finally, we introduce a self-regularization loss that incorporates intrinsic image properties and essential visual attributes to guide the output towards meeting human visual expectations. Comprehensive experiments validate the competitiveness of our proposed algorithm in both qualitative and quantitative assessments. Notably, our noise estimation method, with linear time complexity and suitable for various denoisers, significantly improves both denoising and enhancement performance. Benefiting from this, our approach achieves a 0.675dB PSNR improvement on the LOL dataset and 0.818dB on the MIT dataset on LLIE task, even compared to supervised methods. The source code is available at \href{https://doi.org/10.5281/zenodo.11463142}{this DOI repository} and the specific code for noise estimation can be found at \href{https://github.com/GoogolplexGoodenough/noise_estimate}{this separate GitHub link}.

Figures (11)

• Figure 1: Comparison among recent state-of-the-art unsupervised methods, including EnGAN jiang2021enlightengan, ZeroDCE guo2020zero, RetinexDIP zhao2021retinexdip, RUAS liu2021ruas, and SCI ma2022toward. A scaled heatmap showcasing the absolute error (darker is better) and an illumination map are concatenated to the right part of the enhanced image for comprehensive comparison. Our method closely resembles the ground truth and exhibits a globally smooth yet structure-aware illumination map.
• Figure 2: Illustrations of the pipeline and fundamental components. Top: (a) Self-Calibrated Denoiser, (b) Learnable Illumination Interpolator. Bottom: (c) Denoise Module $\mathcal{N}_{d}(\mathbf{x},\bm{\omega_d})$, (d) Illumination Learning Module $\mathcal{M}_{i}(\mathbf{x},\bm{\omega_i})$, (e) Conditional Feature Modulation (CFM).
• Figure 3: Visualization of intermediate results (i.e., $\mathbf{x}$, $\mathbf{v}$, $\mathbf{y}$ and $\mathbf{s}$) regarding the workflow based on Equation3 (\ref{['eq:scd-to-lii']}).
• Figure 4: Compared with state-of-the-art unsupervised methods, II, the Illumination Interpolator, has ability to enhance images with different degradation factors, which simply divide the linear interpolation result between the input image and a unit vector.
• Figure 5: Visual comparison of estimated illumination map. Left: Combination reference produced by the ground truth, Right: Illumination estimated via reverse procedure (i.e., $\mathbf{x} \oslash \mathbf{s}$).