HiLLIE: Human-in-the-Loop Training for Low-Light Image Enhancement

TL;DR

This work tackles low-light image enhancement by aligning outputs with human visual preferences through a human-in-the-loop training paradigm. It introduces HiLLIE, which alternates training between an enhancer and a tailored image quality ranker across multiple stages, using dense data selection and small-scale pairwise annotations to learn a perceptual objective. The ranker is trained on stage-generated image pairs to approximate human judgments via a margin-ranking loss, and its predicted quality guides the enhancer through a ranker loss while preserving content with a fidelity constraint. Experiments on LOLv1/LOLv2 and unpaired real-world sets show that HiLLIE improves no-reference IQA metrics and subjective preferences, albeit with a trade-off in pixel-level fidelity (PSNR/SSIM). The approach is the first to apply human-in-the-loop concepts to unsupervised LLIE, and the authors provide datasets and code to facilitate reproducibility and extension to other perceptual tasks.

Abstract

Developing effective approaches to generate enhanced results that align well with human visual preferences for high-quality well-lit images remains a challenge in low-light image enhancement (LLIE). In this paper, we propose a human-in-the-loop LLIE training framework that improves the visual quality of unsupervised LLIE model outputs through iterative training stages, named HiLLIE. At each stage, we introduce human guidance into the training process through efficient visual quality annotations of enhanced outputs. Subsequently, we employ a tailored image quality assessment (IQA) model to learn human visual preferences encoded in the acquired labels, which is then utilized to guide the training process of an enhancement model. With only a small amount of pairwise ranking annotations required at each stage, our approach continually improves the IQA model's capability to simulate human visual assessment of enhanced outputs, thus leading to visually appealing LLIE results. Extensive experiments demonstrate that our approach significantly improves unsupervised LLIE model performance in terms of both quantitative and qualitative performance. The code and collected ranking dataset will be available at https://github.com/LabShuHangGU/HiLLIE.

Figures (11)

• Figure 1: Illustration of the HiLLIE training framework.
• Figure 2: Qualitative comparisons on LOLv1 (top row), LOLv2-real (middle row) and LIME datasets (bottom row).
• Figure 3: Ablation study of enhancer model performance on no-reference IQA metrics across different training stages. More details can be found in Sec. \ref{['sec:ablation']}.
• Figure 4: User Study. Global scores of different LLIE methods. More details can be found in Sec. \ref{['sec:user study']}.
• Figure 5: Prediction accuracy of $g^{(n)}$ on all collected $f^{(n)}$ outputs.