Low-Light Image Enhancement using Event-Based Illumination Estimation

TL;DR

The paper addresses LLIE in extremely low-light scenes by moving from motion-triggered edge cues to temporal-mapping events that encode illumination. It introduces RetinEV, a Retinex-based pipeline with a Time-to-Illumination (T2I) module and an Illumination-aid Reflectance Enhancement (IRE) module, leveraging cross-modal attention to refine reflectance using high-fidelity illumination estimates. A low-light degradation model (LLDM) and a real-world EvLowLight dataset collected with a beam-splitter system are proposed to support training and evaluation. Empirical results on five synthetic datasets and EvLowLight show state-of-the-art performance with real-time inference at 35.6 FPS for 640×480 images, highlighting the practical potential of event cameras for LLIE and HDR imaging.

Abstract

Low-light image enhancement (LLIE) aims to improve the visibility of images captured in poorly lit environments. Prevalent event-based solutions primarily utilize events triggered by motion, i.e., ''motion events'' to strengthen only the edge texture, while leaving the high dynamic range and excellent low-light responsiveness of event cameras largely unexplored. This paper instead opens a new avenue from the perspective of estimating the illumination using ''temporal-mapping'' events, i.e., by converting the timestamps of events triggered by a transmittance modulation into brightness values. The resulting fine-grained illumination cues facilitate a more effective decomposition and enhancement of the reflectance component in low-light images through the proposed Illumination-aided Reflectance Enhancement module. Furthermore, the degradation model of temporal-mapping events under low-light condition is investigated for realistic training data synthesizing. To address the lack of datasets under this regime, we construct a beam-splitter setup and collect EvLowLight dataset that includes images, temporal-mapping events, and motion events. Extensive experiments across 5 synthetic datasets and our real-world EvLowLight dataset substantiate that the devised pipeline, dubbed RetinEV, excels in producing well-illuminated, high dynamic range images, outperforming previous state-of-the-art event-based methods by up to 6.62 dB, while maintaining an efficient inference speed of 35.6 frame-per-second on a 640X480 image.

Figures (6)

• Figure 1: In contrast to existing event-based LLIE methods that rely on motion events liang2024towards, our RetinEV leverages temporal mapping events for illumination estimation and reflectance decomposition, leading to better visibility.
• Figure 2: The architecture of our RetinEV.$\mathcal{F}_{Decom}$ for low-light images and normal-light images share the same weights. $\beta$ is added for brightness manipulation. "IRE": Illumination-aid Reflectance Enhancement, "$S_{low}$": low-light image, "$S_{normal}$": normal-light image, "$I$": illumination component, "$R$": Reflectance component, "$\hat{S}_{low}$": the predicted enlightened image.
• Figure 3: Visualization of the effect of $\beta$ on the brightness of I and the result.
• Figure 4: Details about EvLowLight dataset. The beam splitter is positioned between the lens and sensors.
• Figure 5: Visual comparison on LOL-v1 wei2018deep dataset. Our method effectively enhances visibility and preserves fine-grained textures.