Seeing the Unseen: Zooming in the Dark with Event Cameras

TL;DR

This work tackles low-light video super-resolution (LVSR) by introducing RetinexEVSR, an event-driven LVSR framework that leverages high-contrast event signals and Retinex-inspired priors. A bidirectional cross-modal fusion (RBF) enables mutual guidance between degraded RGB frames and events, while the Illumination-guided Event Enhancement (IEE) and Event-guided Reflectance Enhancement (ERE) modules progressively refine event and reflectance features to support high-quality upsampling. The method achieves state-of-the-art performance on three datasets, with notable gains in PSNR and reductions in runtime compared to prior event-based approaches, and demonstrates strong generalization to real-world low-light scenarios. This approach enhances night-time video tasks such as surveillance and night videography by producing well-lit, texture-rich HR video from challenging LR inputs.

Abstract

This paper addresses low-light video super-resolution (LVSR), aiming to restore high-resolution videos from low-light, low-resolution (LR) inputs. Existing LVSR methods often struggle to recover fine details due to limited contrast and insufficient high-frequency information. To overcome these challenges, we present RetinexEVSR, the first event-driven LVSR framework that leverages high-contrast event signals and Retinex-inspired priors to enhance video quality under low-light scenarios. Unlike previous approaches that directly fuse degraded signals, RetinexEVSR introduces a novel bidirectional cross-modal fusion strategy to extract and integrate meaningful cues from noisy event data and degraded RGB frames. Specifically, an illumination-guided event enhancement module is designed to progressively refine event features using illumination maps derived from the Retinex model, thereby suppressing low-light artifacts while preserving high-contrast details. Furthermore, we propose an event-guided reflectance enhancement module that utilizes the enhanced event features to dynamically recover reflectance details via a multi-scale fusion mechanism. Experimental results show that our RetinexEVSR achieves state-of-the-art performance on three datasets. Notably, on the SDSD benchmark, our method can get up to 2.95 dB gain while reducing runtime by 65% compared to prior event-based methods. Code: https://github.com/DachunKai/RetinexEVSR.

Figures (16)

• Figure 1: An example (a) from an extremely low-light (-6.7 EV) LR sample, enhanced by (b) SOTA LVE li2023fastllve + VSR xu2024enhancing methods; (c) SOTA one-stage LVSR method lu2023learning; and (e) our event-based approach. It can be observed that only our method produces well-lit, high-quality results with clearly recognizable text.
• Figure 2: In low light, both RGB and event signals degrade: the RGB frame suffers from severe illumination and detail loss, and the event data contains noise and trailing artifacts.
• Figure 3: Comparison of LVSR strategies. (a) RGB-based method xu2023deep directly super-resolves low-light frames. (b) Previous event-based methods lu2023learningkai2024evtexture directly fuse two degraded modalities. (c) Our RBF strategy first uses illumination to guide event refinement and then leverages the refined events to enhance reflectance, enabling effective information integration.
• Figure 4: Network architecture of RetinexEVSR. (a) The model takes low-light LR frames and corresponding events as input, and outputs HR frames with well-lit details. Each frame is decomposed into illumination and reflectance, and optical flow is estimated from reflectance for temporal alignment. (b) At each time step, the IEE module uses illumination to guide event enhancement. (c) The refined event features are then used in the ERE module to enhance reflectance features.
• Figure 5: Qualitative comparison on RELED for 4$\times$ LVSR. The bottom row is the statistical distribution of the RGB channels. Our method recovers clearer license plate numbers and more faithful colors that better match the ground truth.