EVREAL: Towards a Comprehensive Benchmark and Analysis Suite for Event-based Video Reconstruction

TL;DR

This work addresses the lack of standardized benchmarks for event-based video reconstruction by introducing EVREAL, an open-source framework that unifies evaluation protocols and benchmarks across diverse datasets. It provides a comprehensive analysis of state-of-the-art methods (e.g., E2VID, FireNet, SPADE-E2VID, ET-Net) under varying conditions and includes both quantitative metrics and computational complexity considerations, as well as qualitative visualizations. The results offer actionable insights into how methods perform under different motion, lighting, and downstream-task scenarios, enabling fairer comparisons and guiding future design of robust event-based reconstruction algorithms. Overall, EVREAL aims to accelerate progress in the field by offering standardized, extensible evaluation, detailed methodological analysis, and practical benchmarks for real-world deployment.

Abstract

Event cameras are a new type of vision sensor that incorporates asynchronous and independent pixels, offering advantages over traditional frame-based cameras such as high dynamic range and minimal motion blur. However, their output is not easily understandable by humans, making the reconstruction of intensity images from event streams a fundamental task in event-based vision. While recent deep learning-based methods have shown promise in video reconstruction from events, this problem is not completely solved yet. To facilitate comparison between different approaches, standardized evaluation protocols and diverse test datasets are essential. This paper proposes a unified evaluation methodology and introduces an open-source framework called EVREAL to comprehensively benchmark and analyze various event-based video reconstruction methods from the literature. Using EVREAL, we give a detailed analysis of the state-of-the-art methods for event-based video reconstruction, and provide valuable insights into the performance of these methods under varying settings, challenging scenarios, and downstream tasks.

Figures (6)

• Figure 1: Additional qualitative comparisons on the ECD dataset.
• Figure 2: Additional qualitative comparisons on the MVSEC dataset.
• Figure 3: Additional qualitative comparisons on the HQF dataset.
• Figure 4: Additional qualitative comparisons on the BS-ERGB dataset.
• Figure 5: Additional qualitative comparisons on the fast parts of the ECD dataset (ECD-FAST).