DeblurGS: Gaussian Splatting for Camera Motion Blur

TL;DR

This work tackles the challenge of reconstructing sharp 3D scenes from motion-blurred multi-view images when initial camera poses are noisy, a common scenario for SfM with blurry inputs. It introduces DeblurGS, a 3D Gaussian Splatting framework that jointly optimizes a sharp 3DGS and latent 6-DOF camera motion by synthesizing blurry renderings from estimated trajectories and comparing them to observed blur, aided by a Gaussian Densification Annealing scheme. The method includes a sub-frame alignment mechanism to account for sampling variability along the camera trajectory and a gamma-correction step to reflect camera response, with a temporal smoothness loss to stabilize optimization. Experiments on real and synthetic benchmarks, including field smartphone videos, show DeblurGS achieves state-of-the-art deblurring and novel view synthesis, demonstrating strong robustness to pose noise and practical applicability in real-world scenarios. The approach eliminates the need for pre-trained 2D deblurring networks, enabling high-fidelity 3D reconstructions directly from blurry inputs.

Abstract

Although significant progress has been made in reconstructing sharp 3D scenes from motion-blurred images, a transition to real-world applications remains challenging. The primary obstacle stems from the severe blur which leads to inaccuracies in the acquisition of initial camera poses through Structure-from-Motion, a critical aspect often overlooked by previous approaches. To address this challenge, we propose DeblurGS, a method to optimize sharp 3D Gaussian Splatting from motion-blurred images, even with the noisy camera pose initialization. We restore a fine-grained sharp scene by leveraging the remarkable reconstruction capability of 3D Gaussian Splatting. Our approach estimates the 6-Degree-of-Freedom camera motion for each blurry observation and synthesizes corresponding blurry renderings for the optimization process. Furthermore, we propose Gaussian Densification Annealing strategy to prevent the generation of inaccurate Gaussians at erroneous locations during the early training stages when camera motion is still imprecise. Comprehensive experiments demonstrate that our DeblurGS achieves state-of-the-art performance in deblurring and novel view synthesis for real-world and synthetic benchmark datasets, as well as field-captured blurry smartphone videos.

Figures (7)

• Figure 1: Novel View Synthesis with Blurry Views. Our DeblurGS achieves state-of-the-art performance in novel view synthesis and deblurring compared to previous approaches.
• Figure 2: Training pipeline of DeblurGS. We simulate the physical blur operation while the camera is moving. In our optimization, the blurry images $\{\hat{\mathbf{B}_i}\}_{i=1}^{M}$ are reconstructed by accumulating rendered images along with the estimated camera trajectories. We minimize L1 loss $\mathcal{L}_\text{rgb}$ between the input blurry images $\{\mathbf{B}_i\}_{i=1}^{M}$ and reconstructed blurry images $\{\hat{\mathbf{B}_i}\}_{i=1}^{M}$ to jointly optimize the camera motion trajectories and the sharp 3D scene.
• Figure 3: Illustration of the Sub-frame Alignment Parameters. With the estimated camera trajectory, the resulting blurry image changes based on the sampling intervals of sub-frame images. Even if the latent camera trajectory is well-optimized, the evenly sampled blurry image ($\mathbf{1^{st}}$ row) does not correspond to the blurry observation of real data. We align the sub-frames intervals by optimizing the sub-frame alignment parameters to accumulate precise blurry image ($\mathbf{2^{nd}}$row).
• Figure 4: Qualitative Comparison of Deblurring on Real Motion BlurMa_deblurnerf. Note that blurry views in Real Motion Blur do not have their ground-truth pairs.
• Figure 5: Qualitative Comparison of Deblurring on ExBlur-CPlee2023exblurf. The camera poses are initialized from COLMAP schonberger2016structure with sharp pairs corresponding to each blurry observation.