Unsupervised Low-light Image Enhancement with Lookup Tables and Diffusion Priors

TL;DR

The paper tackles the challenge of achieving high-quality low-light image enhancement in real time without requiring paired training data. It introduces DPLUT, a two-LUT framework comprising a light adjustment LUT (LLUT) for coarse, image-adaptive curve-based enhancement and a noise suppression LUT (NLUT) that leverages diffusion priors to suppress amplified noise. LLUT is trained with unsupervised losses to produce pixel-wise curve parameters, while NLUT is guided by a pretrained diffusion model to generate a pseudo-reference for noise mitigation. Extensive experiments on LOL, SICE-Part2, and LSRW demonstrate that DPLUT delivers state-of-the-art performance among unsupervised methods and competitive results with supervised approaches, while enabling real-time processing of 4K images. The approach promises practical impact for ISP pipelines and resource-constrained devices, with potential extensions to joint LUT optimization and broader vision tasks.

Abstract

Low-light image enhancement (LIE) aims at precisely and efficiently recovering an image degraded in poor illumination environments. Recent advanced LIE techniques are using deep neural networks, which require lots of low-normal light image pairs, network parameters, and computational resources. As a result, their practicality is limited. In this work, we devise a novel unsupervised LIE framework based on diffusion priors and lookup tables (DPLUT) to achieve efficient low-light image recovery. The proposed approach comprises two critical components: a light adjustment lookup table (LLUT) and a noise suppression lookup table (NLUT). LLUT is optimized with a set of unsupervised losses. It aims at predicting pixel-wise curve parameters for the dynamic range adjustment of a specific image. NLUT is designed to remove the amplified noise after the light brightens. As diffusion models are sensitive to noise, diffusion priors are introduced to achieve high-performance noise suppression. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods in terms of visual quality and efficiency.

Figures (10)

• Figure 1: Comparisons of performance and efficiency. The average PSNR is evaluated on LSRW LSRW, and inference time is evaluated on 4K ($3840\times 2160$) resolution with a single Titan RTX GPU. Our approach obtains the highest PSNR and can process 4K low-light images in real-time. Note that $^{\star}$ and $^{\star}$ indicate the maximum size that the models can handle is 480P ($640\times 480$) and 1080P ($1920\times 1080$), respectively.
• Figure 2: Visual performance in real-world scenes with 4K resolution. Our DPLUT achieves visually pleasing results in terms of brightness, color, contrast, and naturalness across diverse scenes and under various light distributions.
• Figure 3: The overall framework of our proposed DPLUT. In the training phase, DPLUT involves two main stages. (a) In the first stage, we learn a light adjustment lookup table (LLUT), which estimates pixel-wise curve parameters for yielding coarse normal-light images. (b) In the second stage, we learn a noise suppression lookup table (NLUT) by introducing knowledge of a diffusion model, aiming at removing the amplified noise and artifacts introduced from LLUT. In the testing phase, with the LLUT and NLUT, DPLUT can robustly recover perceptual-friendly results in real-time.
• Figure 4: (a) The architecture of our two key components: a light adjustment lookup table (LLUT) and a noise suppression lookup table (NLUT). (b) We applied an inverse discrete Fourier transform to the phase of the low/normal-light image to obtain the phase-only reconstruction image (PCI) in the spatial domain. That means the amplitude of low/normal-light image is set to 1.
• Figure 5: Visual comparisons of the ablation study. The full model achieves the best performance.