Deblur Gaussian Splatting SLAM

TL;DR

Deblur-SLAM addresses the challenge of reconstructing sharp 3D scenes from motion-blurred RGB sequences by explicitly modeling the image formation during exposure. It unifies frame-to-frame tracking with a frame-to-model tracker on a deformable 3D Gaussian representation, rendering sharp sub-frames and matching their average blur to observations. Online loop closure and global bundle adjustment ensure global consistency, aided by a monocular depth estimator for robust map initialization. Empirically, it achieves state-of-the-art sub-frame trajectory recovery and sharp map estimates on synthetic Replica data and real-world datasets (ScanNet, TUM-RGBD, ETH3D), outperforming baselines in PSNR, SSIM, LPIPS, and ATE. This approach broadens RGB-only SLAM to highly blurred sequences, enabling robust, high-fidelity mapping for robotics and AR/VR in motion-rich environments.

Abstract

We present Deblur-SLAM, a robust RGB SLAM pipeline designed to recover sharp reconstructions from motion-blurred inputs. The proposed method bridges the strengths of both frame-to-frame and frame-to-model approaches to model sub-frame camera trajectories that lead to high-fidelity reconstructions in motion-blurred settings. Moreover, our pipeline incorporates techniques such as online loop closure and global bundle adjustment to achieve a dense and precise global trajectory. We model the physical image formation process of motion-blurred images and minimize the error between the observed blurry images and rendered blurry images obtained by averaging sharp virtual sub-frame images. Additionally, by utilizing a monocular depth estimator alongside the online deformation of Gaussians, we ensure precise mapping and enhanced image deblurring. The proposed SLAM pipeline integrates all these components to improve the results. We achieve state-of-the-art results for sharp map estimation and sub-frame trajectory recovery both on synthetic and real-world blurry input data.

Figures (6)

• Figure 1: Deblur-SLAM can successfully track the camera and reconstruct sharp maps for highly motion-blurred sequences. We directly model motion blur, which enables us to achieve high-quality reconstructions, both on challenging synthetic (top) and real (bottom) data.
• Figure 2: Architecture of Deblur-SLAM. Given an RGB input stream, we estimate an initial pose through local bundle adjustment (BA) using joint Disparity, Scale and Pose Optimization (DSPO). This pose is later refined through frame-to-model tracking that learns a sub-frame trajectory. Each keyframe is then mapped, taking advantage of the estimated monocular depth. The sub-frame trajectory is applied to render virtual sharp images, which model the physical image formation of blurry images. We optimize the photometric and geometric error between the observed blurry image and the average of our sharp images. We further refine poses globally via online loop closure, global BA, and a deformable 3D Gaussian map that adjusts for global pose and depth updates before each mapping phase.
• Figure 3: Qualitative results on real-world ScanNet Dai2017ScanNet data. Given the input blurry frames from scenes 0169 and 0207, we manage to track the trajectory with sub-frame precision and estimate sharp maps by directly modeling the camera motion blur.
• Figure 4: Qualitative comparison to Splat-SLAM. We compare in both a darker scene (office 1 -- top) and a very bright scene (office 2 -- bottom) on the newly proposed synthetically blurred Replica dataset straub2019replica, showing the versatility of the proposed model. From left to right: the observed frame with visible camera motion blur, the SplatSLAM sandstrom2024splat rendering, one of our sharp virtual images, and the ground truth. The proposed Deblur-SLAM method can recover more details such as the boxes (top) and the pillow (bottom).
• Figure 5: Qualitative results on the Replica dataset (office 3 scene) straub2019replica. We outperform Splat-SLAM sandstrom2024splat on blurry data.