LED: A Large-scale Real-world Paired Dataset for Event Camera Denoising

TL;DR

This work addresses the challenge of real-world event camera noise by introducing LED, a large-scale paired dataset with diverse noise levels, illumination, and high-resolution data to support robust denoising. It proposes DED, a dual-events denoising framework that leverages noise inconsistency and signal consistency to generate high-quality ground truth, and DTSNN, a fully spiking neural network with a dynamic threshold mechanism for efficient, accurate denoising. Experiments demonstrate that LED enables superior GT quality and generalization, with DTSNN achieving state-of-the-art denoising performance and energy efficiency across LED and public datasets. The dataset and code release are positioned to accelerate realistic event-based denoising research and applications.

Abstract

Event camera has significant advantages in capturing dynamic scene information while being prone to noise interference, particularly in challenging conditions like low threshold and low illumination. However, most existing research focuses on gentle situations, hindering event camera applications in realistic complex scenarios. To tackle this limitation and advance the field, we construct a new paired real-world event denoising dataset (LED), including 3K sequences with 18K seconds of high-resolution (1200*680) event streams and showing three notable distinctions compared to others: diverse noise levels and scenes, larger-scale with high-resolution, and high-quality GT. Specifically, it contains stepped parameters and varying illumination with diverse scenarios. Moreover, based on the property of noise events inconsistency and signal events consistency, we propose a novel effective denoising framework(DED) using homogeneous dual events to generate the GT with better separating noise from the raw. Furthermore, we design a bio-inspired baseline leveraging Leaky-Integrate-and-Fire (LIF) neurons with dynamic thresholds to realize accurate denoising. The experimental results demonstrate that the remarkable performance of the proposed approach on different datasets.The dataset and code are at https://github.com/Yee-Sing/led.

Figures (10)

• Figure 1: Illustration of the proposed dataset LED. (a) Distribution of noise level and scene of the proposed dataset. (b) Our proposed LED outperforms others in terms of sequences, capture, and resolution (Circles with numbers to indicate). (c) LED collects diverse event streams across various conditions of illumination, depth of field, and target scale.
• Figure 2: The substantial noise distinctions between frame/event camera. (a) For a stationary grayscale chart, the frame camera sequentially acquires noisy samples, fluctuating around a latent value within a specific distribution. (b) In a horizontal moving case, the event camera outputs binary signal serially, comprising signal events from motion gradient edges and spurious BA noise.
• Figure 3: Illustration of event camera dual-sampling analysis: (a) Given an ideal intensity input, which includes varying and steady stages, generates a series of signal events. (b) The twice samplings of the same input in the circuit model, both generate additional BA noise in previously steady stages, resulting in consistency discrepancies between noise events and signal events. (c) The visualization results of the actual dual-sampled events cumulative frame demonstrate the misalignment of inconsistent dual-sampled noise events and the alignment of coexisting signal events (two color groups indicating the respective polarities of the dual events). (d) The statistical results of the two tests also prove a low overlap rate between dual-sampled event pixels and a much higher one where signal events are present.
• Figure 4: Overview of the dual events denoising framework. (a) Our collection device consists of two identical EVK4s forming a co-axial system with a 1:1 beamsplitter. (b) We first perform spatial similarity processing to retain the consistent parts, followed by sequentially spatiotemporal correlation constraints to remove the residual small amount of noise event from the previous step.
• Figure 5: Analysis of different event denoising results on our dataset. From left to right, the first column is the raw events, and the remaining five columns represent different methods, namely DWF, STDF, TimeSurface, EvFlow, and the proposed DED. From top to bottom, the first row shows the denoised results, the second row is the residual noise, the third row denotes the statistical distribution of the denoised results and the last row represents the intensity image reconstruction corresponding to the zoomed region denoised events.