LoLI-Street: Benchmarking Low-Light Image Enhancement and Beyond

TL;DR

A new dataset LoLI-Street (Low-Light Images of Streets) with 33k paired low-light and well-exposed images from street scenes in developed cities, covering 19k object classes for object detection and a transformer and diffusion-based LLIE model named"TriFuse" is introduced.

Abstract

Low-light image enhancement (LLIE) is essential for numerous computer vision tasks, including object detection, tracking, segmentation, and scene understanding. Despite substantial research on improving low-quality images captured in underexposed conditions, clear vision remains critical for autonomous vehicles, which often struggle with low-light scenarios, signifying the need for continuous research. However, paired datasets for LLIE are scarce, particularly for street scenes, limiting the development of robust LLIE methods. Despite using advanced transformers and/or diffusion-based models, current LLIE methods struggle in real-world low-light conditions and lack training on street-scene datasets, limiting their effectiveness for autonomous vehicles. To bridge these gaps, we introduce a new dataset LoLI-Street (Low-Light Images of Streets) with 33k paired low-light and well-exposed images from street scenes in developed cities, covering 19k object classes for object detection. LoLI-Street dataset also features 1,000 real low-light test images for testing LLIE models under real-life conditions. Furthermore, we propose a transformer and diffusion-based LLIE model named "TriFuse". Leveraging the LoLI-Street dataset, we train and evaluate our TriFuse and SOTA models to benchmark on our dataset. Comparing various models, our dataset's generalization feasibility is evident in testing across different mainstream datasets by significantly enhancing images and object detection for practical applications in autonomous driving and surveillance systems. The complete code and dataset is available on https://github.com/tanvirnwu/TriFuse.

Figures (6)

• Figure 1: Comparison between our TriFuse and SOTA models using a sample real low-light test image. (a) Qualitative comparison: Visually, PairLIE and RQ-LLIE produce brighter outputs but lack realism. In contrast, TriFuse ensures high visual quality with realistic enhancements. (b) Quantitative comparison based on the no-reference metric BRISQUE ($\downarrow$) and inference time ($\downarrow$).
• Figure 2: (a) Sample images of LoLI-Street. Green: train and validation sets, Red: real low-light test set. (b) Distribution of the low-light images of our LoLI-Street.
• Figure 3: Overview of our TriFuse model, featuring the Conditional Noise Module (CNM) and Edge Sharpening Module (ESM) for effective LLIE. The CNM generates noise, refined through forward and backward passes by the diffusion process. The ESM sharpens the output image's edges. The process starts with predicted noise, undergoes wavelet transformation and denoising within TriFuse, and results in a visually enhanced image.
• Figure 4: Enhanced images by models picking a random image from the (a) synthetic validation set and (b) real low-light test set of our LoLI-Street dataset.
• Figure 5: Enhanced images by SOTA models and our proposed TriFuse picking a random image from the validation sets of mainstream LLIE datasets.