BeSplat: Gaussian Splatting from a Single Blurry Image and Event Stream

TL;DR

BeSplat tackles motion blur in single-image 3D reconstruction by jointly optimizing a Gaussian Splatting scene representation and a Bézier SE(3) camera trajectory using a single blurred image and its event stream. It leverages the explicit, differentiable 3D Gaussian representation for fast rendering and combines a photo-formation loss with an event-consistency loss to recover sharp views along the unknown camera path. The method achieves accelerated training and real-time rendering compared to NeRF-based approaches while maintaining high reconstruction quality on synthetic and real data. The key contribution is integrating temporal event data into a Gaussian Splatting framework to resolve a highly ill-posed single-image deblurring problem and recover both geometry and motion.

Abstract

Novel view synthesis has been greatly enhanced by the development of radiance field methods. The introduction of 3D Gaussian Splatting (3DGS) has effectively addressed key challenges, such as long training times and slow rendering speeds, typically associated with Neural Radiance Fields (NeRF), while maintaining high-quality reconstructions. In this work (BeSplat), we demonstrate the recovery of sharp radiance field (Gaussian splats) from a single motion-blurred image and its corresponding event stream. Our method jointly learns the scene representation via Gaussian Splatting and recovers the camera motion through Bezier SE(3) formulation effectively, minimizing discrepancies between synthesized and real-world measurements of both blurry image and corresponding event stream. We evaluate our approach on both synthetic and real datasets, showcasing its ability to render view-consistent, sharp images from the learned radiance field and the estimated camera trajectory. To the best of our knowledge, ours is the first work to address this highly challenging ill-posed problem in a Gaussian Splatting framework with the effective incorporation of temporal information captured using the event stream.

Figures (5)

• Figure 1: Given a single blurry image and its corresponding event stream, BeSplat synthesizes high-quality, novel sharp images along the camera trajectory by harnessing event information to precisely estimate motion trajectory and restore fine details with clarity.
• Figure 2: Our Method: The BeSplat framework reconstructs sharp radiance fields (Gaussian splats) while accurately estimating the camera motion trajectory, modeled with a Bézier curve, from a single blurry image and its corresponding event stream. The framework jointly optimizes the Gaussian splats and camera motion trajectory by integrating RGB loss to align the synthesized and captured blurry images, as well as event loss to ensure consistency between the synthesized and captured events along the trajectory.
• Figure 3: Results on the Synthetic Dataset: Our method consistently achieves high performance, closely rivaling BeNeRF in image reconstruction quality. Insets highlight specific regions of the images, demonstrating that our method achieves comparable visual fidelity to BeNeRF while offering significant reductions in training and rendering times, as well as lower GPU memory usage.
• Figure 4: Results on the Real Dataset: Our method effectively handles varying levels of blur, consistently producing sharp, high-quality images. Insets illustrate that while our approach delivers results comparable to state-of-the-art methods on synthetic datasets, it performs noticeably better on real-world data by providing sharper and cleaner details, highlighting its robustness and reliability.
• Figure 5: Ablation Studies on Virtual Camera Count: Our method demonstrates robust deblurring performance, maintaining high-quality results even when using a limited number of virtual cameras along the camera motion trajectory.