Event3DGS: Event-Based 3D Gaussian Splatting for High-Speed Robot Egomotion

TL;DR

Event3DGS targets high-speed robot egomotion where frame-based 3D reconstruction suffers from motion blur. It extends 3D Gaussian Splatting to operate on event streams by integrating the event formation model into a differentiable rendering framework, using neutralization-aware slicing, sparsity-aware sampling, a specialized event rendering loss, and progressive training. The method also includes blur-aware rasterization and a parameter-separable refinement to leverage a small number of motion-blurred RGB images for appearance fidelity. Empirical results show substantial gains in reconstruction quality (average PSNR improvements of multiple dB) and dramatic efficiency (orders of magnitude faster training and rendering) compared to baselines, enabling closer to real-time dense 3D reconstruction in dynamic robotics tasks.

Abstract

By combining differentiable rendering with explicit point-based scene representations, 3D Gaussian Splatting (3DGS) has demonstrated breakthrough 3D reconstruction capabilities. However, to date 3DGS has had limited impact on robotics, where high-speed egomotion is pervasive: Egomotion introduces motion blur and leads to artifacts in existing frame-based 3DGS reconstruction methods. To address this challenge, we introduce Event3DGS, an {\em event-based} 3DGS framework. By exploiting the exceptional temporal resolution of event cameras, Event3GDS can reconstruct high-fidelity 3D structure and appearance under high-speed egomotion. Extensive experiments on multiple synthetic and real-world datasets demonstrate the superiority of Event3DGS compared with existing event-based dense 3D scene reconstruction frameworks; Event3DGS substantially improves reconstruction quality (+3dB) while reducing computational costs by 95\%. Our framework also allows one to incorporate a few motion-blurred frame-based measurements into the reconstruction process to further improve appearance fidelity without loss of structural accuracy.

Figures (11)

• Figure 1: Left: Conventional (frame-based) 3D Gaussian Splatting fails to reconstruct geometric details due to motion blur caused by high-speed robot egomotion. Right: By exploiting the high temporal resolution of event cameras, Event3DGS can effectively reconstruct structure and appearance in the presence of fast egomotion.
• Figure 2: Event3DGS Architecture. We first utilize a neutralization-aware accumulator (for mitigating the cancellation of positive and negative events) and sparsity-aware sampling strategy (for reconstruction in non-event regions) to process the input event stream into frames. Then, the sampled event frames are utilized as differential supervision between the corresponding rendered views, optimizing the 3D Gaussians to reconstruct sharp structures and apperance from fast egomotion. We train Event3DGS progressively to better represent geometric details. As an optional component, we integrate a few motion-blurred RGB images from an attached frame-based camera into the pipeline. By embedding motion blur formation into the rasterizer and employing a parameter-separable refinement strategy, we calibrate the colorization while preserving structural details.
• Figure 3: Visualization on low-light experimental scenes (event-only).
• Figure 4: Visualization on synthetic and real-world scenes (events emulated from RGB frames). Event3DGS excels in reconstructing sharp structures and appearance details.
• Figure 5: Ablations on progressive training (event-only). The PSNR we report is for the single image. With the pretrained Gaussians as initialization, Event3DGS is able to progressively recover the fine-grained structural details that are under-reconstructed in the $1^{st}$ round training.