LoopSplat: Loop Closure by Registering 3D Gaussian Splats

TL;DR

LoopSplat tackles the global consistency challenge in dense RGB-D SLAM by introducing a coupled 3D Gaussian Splatting system that builds submap-based representations, performs frame-to-model tracking, and enforces loop closure online. It registers Gaussian splats directly between overlapping submaps to generate loop-edge constraints and integrates them into a robust pose-graph optimization, achieving globally consistent reconstructions without saving all input frames. The approach demonstrates competitive or superior tracking, reconstruction, and rendering across Replica, TUM-RGBD, ScanNet, and ScanNet++ datasets, with favorable memory and runtime characteristics. These results establish 3DGS as a unified representation for tracking, mapping, and global consistency in dense SLAM, and the work points to future improvements in mesh extraction and uncertainty modeling for overlaps.

Abstract

Simultaneous Localization and Mapping (SLAM) based on 3D Gaussian Splats (3DGS) has recently shown promise towards more accurate, dense 3D scene maps. However, existing 3DGS-based methods fail to address the global consistency of the scene via loop closure and/or global bundle adjustment. To this end, we propose LoopSplat, which takes RGB-D images as input and performs dense mapping with 3DGS submaps and frame-to-model tracking. LoopSplat triggers loop closure online and computes relative loop edge constraints between submaps directly via 3DGS registration, leading to improvements in efficiency and accuracy over traditional global-to-local point cloud registration. It uses a robust pose graph optimization formulation and rigidly aligns the submaps to achieve global consistency. Evaluation on the synthetic Replica and real-world TUM-RGBD, ScanNet, and ScanNet++ datasets demonstrates competitive or superior tracking, mapping, and rendering compared to existing methods for dense RGB-D SLAM. Code is available at loopsplat.github.io.

Figures (7)

• Figure 1: Dense Reconstruction on ScanNet dai2017scannetscene0054. LoopSplat demonstrates superior performance in geometric accuracy, robust tracking, and high-quality re-rendering. This is enabled by our globally consistent reconstruction approach utilizing 3DGS kerbl20233d.
• Figure 2: LoopSplat Overview. LoopSplat is a coupled RGB-D SLAM system that uses Gaussian splats as a unified scene representation for tracking, mapping, and maintaining global consistency. In the front-end, it continuously estimates the camera position while constructing the scene using Gaussian splats. When the camera traverses beyond a predefined threshold, the current submap is finalized, and a new one is initiated. Concurrently, the back-end loop closure module monitors for location revisits. Upon detecting a loop, the system generates a pose graph, incorporating loop edge constraints derived from our proposed 3DGS registration. Subsequently, pose graph optimization (PGO) is executed to refine both camera poses and submaps, ensuring overall spatial coherence.
• Figure 3: Comparison of Submap Alignment on ScanNet dai2017scannet. We visualize the centers of 3D Gaussians as point clouds. Two submaps are colorized differently. LoopSplat consistently aligns the submaps better than Gaussian-SLAM yugay2023gaussian.
• Figure 4: Comparison of Mesh Reconstruction on two ScanNet dai2017scannet scenes. For the first scene, we highlight shape details with normal shading, showing that LoopSplat yields the best geometry (e.g. the chairs). For the second one, we display renderings of the colored mesh. Note the distortions at the desk in ground truth that are not present in ours, indicating accuracy limitations of ScanNet ground truth poses.
• Figure G.1: Qualitative Results of Overlap Ratio between Submaps. We visualize the centers of 3D Gaussians as point clouds, with two submaps only colorized in the overlapping region. The top row demonstrates a large overlap between submaps with $\mathrm{OR}=0.9$. The middle row showcases a medium overlap of $\mathrm{OR}=0.6$, while the bottom row exhibits an extremely low overlap of $\mathrm{OR}=0.1$. This last case was rejected as a loop due to its insufficient overlap, which typically leads to low-accuracy registration or even complete failure.