Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

E2GS: Event Enhanced Gaussian Splatting

Hiroyuki Deguchi, Mana Masuda, Takuya Nakabayashi, Hideo Saito

TL;DR

This paper introduces Event Enhanced Gaussian Splatting (E2GS), a novel method that incorporates event data into Gaussian Splatting, which has recently made significant advances in the field of novel view synthesis.

Abstract

Event cameras, known for their high dynamic range, absence of motion blur, and low energy usage, have recently found a wide range of applications thanks to these attributes. In the past few years, the field of event-based 3D reconstruction saw remarkable progress, with the Neural Radiance Field (NeRF) based approach demonstrating photorealistic view synthesis results. However, the volume rendering paradigm of NeRF necessitates extensive training and rendering times. In this paper, we introduce Event Enhanced Gaussian Splatting (E2GS), a novel method that incorporates event data into Gaussian Splatting, which has recently made significant advances in the field of novel view synthesis. Our E2GS effectively utilizes both blurry images and event data, significantly improving image deblurring and producing high-quality novel view synthesis. Our comprehensive experiments on both synthetic and real-world datasets demonstrate our E2GS can generate visually appealing renderings while offering faster training and rendering speed (140 FPS). Our code is available at https://github.com/deguchihiroyuki/E2GS.

E2GS: Event Enhanced Gaussian Splatting

TL;DR

This paper introduces Event Enhanced Gaussian Splatting (E2GS), a novel method that incorporates event data into Gaussian Splatting, which has recently made significant advances in the field of novel view synthesis.

Abstract

Event cameras, known for their high dynamic range, absence of motion blur, and low energy usage, have recently found a wide range of applications thanks to these attributes. In the past few years, the field of event-based 3D reconstruction saw remarkable progress, with the Neural Radiance Field (NeRF) based approach demonstrating photorealistic view synthesis results. However, the volume rendering paradigm of NeRF necessitates extensive training and rendering times. In this paper, we introduce Event Enhanced Gaussian Splatting (E2GS), a novel method that incorporates event data into Gaussian Splatting, which has recently made significant advances in the field of novel view synthesis. Our E2GS effectively utilizes both blurry images and event data, significantly improving image deblurring and producing high-quality novel view synthesis. Our comprehensive experiments on both synthetic and real-world datasets demonstrate our E2GS can generate visually appealing renderings while offering faster training and rendering speed (140 FPS). Our code is available at https://github.com/deguchihiroyuki/E2GS.
Paper Structure (15 sections, 12 equations, 5 figures, 5 tables)

This paper contains 15 sections, 12 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. RELATED WORKS
  3. 3D Scene Reconstruction
  4. Event-based 3D Scene Reconstruction
  5. METHOD
  6. Preliminary
  7. Preprocessing
  8. Loss function
  9. EXPERIMENTS
  10. Experimental Setup
  11. Datasets
  12. Quantitative Evaluation
  13. Qualitative Evaluation
  14. Training Time and Rendering Speed
  15. CONCLUSION

Figures (5)

  • Figure 1: When we take as input blurry images of a scene from multiple views, the rendering results of original 3D Gaussian Splatting 3Dgaussians are also severely blurred. In contrast, our E2GS achieves sharper scene rendering by utilizing event data.
  • Figure 2: The overview of the Event Enhanced Gaussian Splatting.
  • Figure 3: Qualiative comparison of the image deblurring task on the real-world dataset.
  • Figure 4: Qualiative comparison of the novel view synthesis task on the real-world dataset.
  • Figure 5: Qualiative comparison on the synthetic dataset. Refer to the red box to see the detailed reconstruction quality. Zoom in for the best view.