Learning to Remove Lens Flare in Event Camera

TL;DR

This work tackles lens flare in event cameras by formulating a physics-grounded forward model that reveals a non-linear, intensity-weighted fusion of background and flare events. It introduces a two-stage physics-driven data generator and a controllable real-world acquisition protocol to create the first paired real-world dataset for this problem, culminating in E-DeflareNet which achieves state-of-the-art restoration. The accompanying E-Deflare Benchmark (E-Flare-2.7K, E-Flare-R, DSEC-Flare) enables robust training, validation, and sim-to-real evaluation, and the approach yields clear improvements in downstream tasks such as event-based imaging and 3D reconstruction. Overall, the paper provides a principled, scalable framework for removing lens flare in event data with broad implications for data-driven restoration in neuromorphic vision.

Abstract

Event cameras have the potential to revolutionize vision systems with their high temporal resolution and dynamic range, yet they remain susceptible to lens flare, a fundamental optical artifact that causes severe degradation. In event streams, this optical artifact forms a complex, spatio-temporal distortion that has been largely overlooked. We present E-Deflare, the first systematic framework for removing lens flare from event camera data. We first establish the theoretical foundation by deriving a physics-grounded forward model of the non-linear suppression mechanism. This insight enables the creation of the E-Deflare Benchmark, a comprehensive resource featuring a large-scale simulated training set, E-Flare-2.7K, and the first-ever paired real-world test set, E-Flare-R, captured by our novel optical system. Empowered by this benchmark, we design E-DeflareNet, which achieves state-of-the-art restoration performance. Extensive experiments validate our approach and demonstrate clear benefits for downstream tasks. Code and datasets are publicly available.

Figures (13)

• Figure 1: Data Generation, Training, and Validation. In the E-Deflare framework, we introduce two parallel pipelines to address data scarcity. The training pipeline synthesizes paired data by first generating dynamic flare events from static images (e.g., Flare7K++) and then fusing them with real background events (e.g., DSEC) via our physics-guided PNL-ES operator. The testing pipeline captures paired real-world data using a controllable optical setup. A removable filter allows us to record the same dynamic scene with flare and without flare, which forms our validation set after post-processing.
• Figure 2: Representative data samples from our benchmark, shown alongside a reference RGB frame (top left). The benchmark includes: a real-world flare example from DSEC-Flare (top right); a synthetic sample from E-Flare-2.7K (bottom left); and a paired real-world sample from our test set E-Flare-R (bottom right).
• Figure 3: Qualitative assessments on E-Flare2.7K. Each column shows the output of a different method applied to the same flare-corrupted input. The rightmost column displays the ground truth for reference. Best viewed in color and zoomed-in for details.
• Figure 4: Qualitative assessments on E-Flare-R. Each column shows the output of a different method applied to the same flare-corrupted input. The rightmost column displays the ground truth for reference. Best viewed in color and zoomed-in for details.
• Figure 5: Downstream task comparisons for event-based imaging. We use SPADE-E2VID cadena2021spade to reconstruct images from both raw and our de-flared event streams on a challenging DSEC scene.