EventSplat: 3D Gaussian Splatting from Moving Event Cameras for Real-time Rendering

TL;DR

EventSplat tackles real-time novel view synthesis from moving event cameras by extending 3D Gaussian Splatting (3DGS) to event data. The method accumulates events into a log-difference image, uses an event-to-video prior to initialize geometry, and employs cubic spline trajectory interpolation to align poses with rapid motion, enabling high-quality renderings at interactive speeds. Key contributions include integrating event accumulation with 3DGS supervision, a robust event-to-video–guided initialization, and trajectory interpolation to stabilize optimization under fast camera motion. The approach achieves state-of-the-art fidelity among event-based view synthesis methods while delivering an order-of-magnitude faster rendering, demonstrating practical viability in challenging lighting and motion conditions.

Abstract

We introduce a method for using event camera data in novel view synthesis via Gaussian Splatting. Event cameras offer exceptional temporal resolution and a high dynamic range. Leveraging these capabilities allows us to effectively address the novel view synthesis challenge in the presence of fast camera motion. For initialization of the optimization process, our approach uses prior knowledge encoded in an event-to-video model. We also use spline interpolation for obtaining high quality poses along the event camera trajectory. This enhances the reconstruction quality from fast-moving cameras while overcoming the computational limitations traditionally associated with event-based Neural Radiance Field (NeRF) methods. Our experimental evaluation demonstrates that our results achieve higher visual fidelity and better performance than existing event-based NeRF approaches while being an order of magnitude faster to render.

Figures (9)

• Figure 1: EventSplat derives 3D representations of scenes in the form of 3D Gaussians from event data, enabling fast real-time renderings of scenes captured with event cameras via rasterization of 3D Gaussians. The use of event-based input is particularly advantageous when traditional RGB cameras fail due to various reasons including poor lighting conditions or motion blur from fast-moving cameras.
• Figure 2: Overview of our 3D Gaussian Splatting training with moving event camera data. Event data streams from $t_{k_{start}}$ to $t_{k_{end}}$ are accumulated into $D$, and distill the point priors from an event-to-video model. The log-difference image $\hat{D}$ is obtained from 3D Gaussian Splatting and Rasterization at two camera view points. It is computed as in \ref{['eq:log_diff']} and remosaicing is performed in case of a color event camera. The respective poses are estimated by cubic spline interpolation.
• Figure 3: Generated Synthetic images comparing our work, event-based NeRF, and E2VID+3DGS qualitatively, with rendering times shown at the bottom of each image. Our work and Robust-e-NeRF outperform others in most scenes. In some cases, e.g., the drums scene, our work performs better and achieves significantly faster rendering times than conventional Robust-e-NeRF composed of InstantNGP.
• Figure 4: Qualitative comparisons of the images generated by our method, event-based NeRF and E2VID+3DGS show that our technique appears to recover details in 5 real scenes. First three rows(07_ziggy_and_fuzz_hdr, 08_peanuts_running and 11_all_characters) are EDS dataset and last two rows are TUM-VIE dataset(mocap-1d-trans and mocap-desk2).
• Figure 5: Qualitative comparisons of the images generated by random initialization and event-to-video guided initialization are shown in 07_ziggy_and_fuzz_hdr scene in EDS dataset.