Equipping Diffusion Models with Differentiable Spatial Entropy for Low-Light Image Enhancement

TL;DR

This work tackles the ill-posed nature of image restoration by shifting from pixel-wise fidelity to distribution-based matching. It introduces differentiable spatial entropy via kernel density estimation to capture both intensity distributions and local spatial structure, including neighborhood information through random-weighted augmentation. By substituting conventional noise-matching losses in diffusion models with an entropy-based objective, the method achieves improved perceptual quality (e.g., lower LPIPS and FID) while retaining fidelity measures like PSNR and SSIM on challenging low-light datasets. The approach demonstrates strong performance on LOL and NTIRE 2024, underscoring the potential of distribution-level losses for realistic, high-quality image restoration in diffusion-based frameworks.

Abstract

Image restoration, which aims to recover high-quality images from their corrupted counterparts, often faces the challenge of being an ill-posed problem that allows multiple solutions for a single input. However, most deep learning based works simply employ l1 loss to train their network in a deterministic way, resulting in over-smoothed predictions with inferior perceptual quality. In this work, we propose a novel method that shifts the focus from a deterministic pixel-by-pixel comparison to a statistical perspective, emphasizing the learning of distributions rather than individual pixel values. The core idea is to introduce spatial entropy into the loss function to measure the distribution difference between predictions and targets. To make this spatial entropy differentiable, we employ kernel density estimation (KDE) to approximate the probabilities for specific intensity values of each pixel with their neighbor areas. Specifically, we equip the entropy with diffusion models and aim for superior accuracy and enhanced perceptual quality over l1 based noise matching loss. In the experiments, we evaluate the proposed method for low light enhancement on two datasets and the NTIRE challenge 2024. All these results illustrate the effectiveness of our statistic-based entropy loss. Code is available at https://github.com/shermanlian/spatial-entropy-loss.

Figures (6)

• Figure 1: Illustration of the traditional fidelity loss (a) and the proposed spatial entropy loss (b). The fidelity loss uses pixel-by-pixel matching for two images while the proposed entropy loss adapts a statistical matching strategy for realistic image generation.
• Figure 2: Overview of the training process for a diffusion model with the proposed spatial entropy loss for noise matching.
• Figure 3: Visual comparison of the proposed model with other low light enhancement approaches on the LoL-v1 wei2018deep dataset.
• Figure 4: Visual comparison of the proposed model with other low light enhancement approaches on the LoL-v2-real yang2020fidelity dataset.
• Figure 5: Visual results of the proposed method on the NTIRE 2024 Low Light Enhancement Challenge dataset.