CaRtGS: Computational Alignment for Real-Time Gaussian Splatting SLAM

TL;DR

CaRtGS addresses the misalignment between computation and resources in Gaussian Splatting SLAM (GS-SLAM) to enable real-time, photorealistic dense scene reconstruction. It introduces adaptive computational alignment composed of fast splat-wise backpropagation, adaptive optimization, and opacity regularization to optimize 3D Gaussian Splatting ($3DGS$) under real-time constraints. The approach yields higher rendering fidelity with far fewer Gaussian primitives and real-time performance across Replica, TUM-RGBD, and VECtor datasets, exceeding $22$ FPS while maintaining accurate localization. The work provides ablative analysis and releases code to support community adoption.

Abstract

Simultaneous Localization and Mapping (SLAM) is pivotal in robotics, with photorealistic scene reconstruction emerging as a key challenge. To address this, we introduce Computational Alignment for Real-Time Gaussian Splatting SLAM (CaRtGS), a novel method enhancing the efficiency and quality of photorealistic scene reconstruction in real-time environments. Leveraging 3D Gaussian Splatting (3DGS), CaRtGS achieves superior rendering quality and processing speed, which is crucial for scene photorealistic reconstruction. Our approach tackles computational misalignment in Gaussian Splatting SLAM (GS-SLAM) through an adaptive strategy that enhances optimization iterations, addresses long-tail optimization, and refines densification. Experiments on Replica, TUM-RGBD, and VECtor datasets demonstrate CaRtGS's effectiveness in achieving high-fidelity rendering with fewer Gaussian primitives. This work propels SLAM towards real-time, photorealistic dense rendering, significantly advancing photorealistic scene representation. For the benefit of the research community, we release the code and accompanying videos on our project website: https://dapengfeng.github.io/cartgs.

Figures (7)

• Figure 1: Performance on TUM-RGBD. We provide a comparison of most of the available open-source GS-SLAM methods.
• Figure 2: The Effect of Adaptive Optimization on Replica. Dashed lines depict performance without adaptive optimization, while solid lines show results with it. Blue represents keyframe iterations, and red indicates PSNR. The horizontal line marks average PSNR and iterations. Our method significantly improves low-PSNR keyframe processing through enhanced iterative optimization, as evident from the trend comparison between dashed and solid lines.
• Figure 3: The overview of CaRtGS. We adopt a real-time cutting-edge SLAM system as a front-end tracker, severing for localization and geometry mapping. In the photorealistic rendering back-end, we apply the proposed adaptive computational alignment strategy to enhance the 3DGS optimization process, including fast splat backward, adaptive optimization, and opacity regularization.
• Figure 4: The Effect of Different Gradient Backpropagation. (a) The original 3DGS employs pixel-wise parallelism for backpropagation, which is prone to frequent contentions, leading to slower backward passes. We introduce a splat-centric parallelism, where each thread handles one Gaussian splat at a time, significantly reducing contention. The gradient computation relies on a set of per-pixel, per-splat values, effectively traversing a splat $\Leftrightarrow$ pixel relationship table. During the forward pass, we save pixel states for every $32^{\text{nd}}$ splat. For the backward pass, splats are grouped into buckets of $32$, each processed by a CUDA warp. Warps utilize intra-warp shuffling to efficiently construct their segment of the state table. (b) We provide a comparison of total iteration on Replica with monocular camera.
• Figure 5: Qualitative results on TUM-RGBD with RGBD Camera.Qualitative assessments demonstrate that our approach significantly improves rendering quality and effectively mitigates visual artifacts. Furthermore, our method achieves precise localization accuracy. In contrast, Gaussian-SLAM exhibits substantial drift, as indicated by the red dashed line.