Ev-GS: Event-based Gaussian splatting for Efficient and Accurate Radiance Field Rendering

TL;DR

Ev-GS addresses the inefficiency and blur issues of frame-based radiance-field rendering by leveraging event cameras within a computational neuromorphic imaging framework. It adapts 3D Gaussian splatting to process purely event-driven signals, linking two viewpoints at timestamps $t$ and $t-w$ through an event-based integral and supervising with $E_{gt}$ and the predicted $E_{pred}$ via linlog and D-SSIM losses. The approach achieves fast training and rendering, dramatically reducing memory usage while maintaining competitive rendering quality relative to frame-based NeRF methods and considerably outperforming Event-NeRF in synthetic tests. This work demonstrates the practicality of efficient, event-based radiance field rendering for real-time, high-dynamic-range scenes and provides a foundation for robust CNI-based view synthesis under challenging lighting and motion conditions.

Abstract

Computational neuromorphic imaging (CNI) with event cameras offers advantages such as minimal motion blur and enhanced dynamic range, compared to conventional frame-based methods. Existing event-based radiance field rendering methods are built on neural radiance field, which is computationally heavy and slow in reconstruction speed. Motivated by the two aspects, we introduce Ev-GS, the first CNI-informed scheme to infer 3D Gaussian splatting from a monocular event camera, enabling efficient novel view synthesis. Leveraging 3D Gaussians with pure event-based supervision, Ev-GS overcomes challenges such as the detection of fast-moving objects and insufficient lighting. Experimental results show that Ev-GS outperforms the method that takes frame-based signals as input by rendering realistic views with reduced blurring and improved visual quality. Moreover, it demonstrates competitive reconstruction quality and reduced computing occupancy compared to existing methods, which paves the way to a highly efficient CNI approach for signal processing.

Figures (4)

• Figure 1: An overview of Ev-GS: a novel method for learning the radiance field volume from a moving event camera via 3D GS (Section \ref{['3D GS']}). We establish a link between the observed events and the rendered views at two distinct timestamps, $t_0$ and $t_1$, utilizing an event-based integral (Section \ref{['Utilization']}), under the supervision of pure event signal (Section \ref{['Supervision']}).
• Figure 2: Demonstration of Hardware Setup.
• Figure 3: Visual Comparison between our approach and Event-NeRF.
• Figure 4: Visual Comparison of real-world data between event signal supervision and frame-based supervision.