Seed Optimization with Frozen Generator for Superior Zero-shot Low-light Enhancement

TL;DR

This paper tackles zero-shot low-light image enhancement by integrating a pre-trained generative model as a reflectance prior within a Retinex-based decomposition. It avoids fine-tuning generator parameters and instead optimizes input seeds, enabling rapid convergence and preservation of rich priors learned from large natural image datasets. The approach is governed by a multi-term loss (reconstruction, illumination regularization, smoothness, and exposure control) and demonstrates strong performance across five benchmarks with favorable visual quality and efficiency. The method, exemplified by using a VQ-VAE-2 pre-trained on FFHQ and ImageNet, offers a general, data-light solution with broad compatibility to different generative priors and practical impact for robust low-light enhancement.

Abstract

In this work, we observe that the generators, which are pre-trained on massive natural images, inherently hold the promising potential for superior low-light image enhancement against varying scenarios.Specifically, we embed a pre-trained generator to Retinex model to produce reflectance maps with enhanced detail and vividness, thereby recovering features degraded by low-light conditions.Taking one step further, we introduce a novel optimization strategy, which backpropagates the gradients to the input seeds rather than the parameters of the low-light enhancement model, thus intactly retaining the generative knowledge learned from natural images and achieving faster convergence speed. Benefiting from the pre-trained knowledge and seed-optimization strategy, the low-light enhancement model can significantly regularize the realness and fidelity of the enhanced result, thus rapidly generating high-quality images without training on any low-light dataset. Extensive experiments on various benchmarks demonstrate the superiority of the proposed method over numerous state-of-the-art methods qualitatively and quantitatively.

Figures (7)

• Figure 1: Visual Comparison on a Representative Low-Light Image. All results are obtained under identical computational resources. As iterative optimization proceeds, Our method generates brighter and more visually appealing results in a shorter time frame compared to two other leading zero-shot low-light enhancement approaches. The image outlined in blue represents the output using our recommended iteration number.
• Figure 2: Retinex decomposition results in different optimization settings. The upper part of the Figure shows the mode of generating reflectance and the lower part shows the results of iterative optimization. The seed-optimization strategy shows better performance in terms of quality. More details will be shown in the supplementary material.
• Figure 3: Overview of Our Proposed Low-Light Enhancement Method. In the preparation phase, we utilize a conventionally trained image generator as the reflectance decoder, while the illumination decoder is initialized randomly. For an input low-light image, our design involves three optimizable seeds: $z_r$, $z_l$, and $\gamma$. $z_r$ and $z_l$ are dedicated to generating a reflectance map rich in detail and a comparatively smoother illumination map, respectively. Concurrently, $\gamma$ is employed for gamma correction of the image. The final enhancement result of the low-light image is achieved solely through iterative optimization of these three inputs.
• Figure 4: Visual quality comparison with several low-light image enhancement state-of-the-art methods. Our approach demonstrates strong detail and color recovery capabilities.
• Figure 5: Comparison of efficiency with other methods that can be trained with a low-light image itself. All results are obtained on the LOL dataset using the same computing resources.