Towards Real-world Event-guided Low-light Video Enhancement and Deblurring
Taewoo Kim, Jaeseok Jeong, Hoonhee Cho, Yuhwan Jeong, Kuk-Jin Yoon
TL;DR
This paper tackles the problem of restoring quality in real-world videos captured under low-light with motion blur by proposing a joint, event-guided approach. It introduces RELED, a real-world dataset collected with a hybrid beam-splitter camera system that provides synchronized low-light blurred frames, normal-light sharp frames, and event streams. The method integrates two novel modules—Event-guided Deformable Temporal Feature Alignment (ED-TFA) and Spectral Filtering-based Cross-Modal Feature Enhancement (SFCM-FE)—to fuse frame and event information for robust, multi-scale restoration. Results show state-of-the-art performance on RELED, surpassing both frame-based and previous event-guided methods, with a lightweight variant offering a favorable accuracy/size trade-off. The work advances practical video enhancement and deblurring in challenging lighting and motion conditions and provides public code for reproducibility.
Abstract
In low-light conditions, capturing videos with frame-based cameras often requires long exposure times, resulting in motion blur and reduced visibility. While frame-based motion deblurring and low-light enhancement have been studied, they still pose significant challenges. Event cameras have emerged as a promising solution for improving image quality in low-light environments and addressing motion blur. They provide two key advantages: capturing scene details well even in low light due to their high dynamic range, and effectively capturing motion information during long exposures due to their high temporal resolution. Despite efforts to tackle low-light enhancement and motion deblurring using event cameras separately, previous work has not addressed both simultaneously. To explore the joint task, we first establish real-world datasets for event-guided low-light enhancement and deblurring using a hybrid camera system based on beam splitters. Subsequently, we introduce an end-to-end framework to effectively handle these tasks. Our framework incorporates a module to efficiently leverage temporal information from events and frames. Furthermore, we propose a module to utilize cross-modal feature information to employ a low-pass filter for noise suppression while enhancing the main structural information. Our proposed method significantly outperforms existing approaches in addressing the joint task. Our project pages are available at https://github.com/intelpro/ELEDNet.