DenseSplat: Densifying Gaussian Splatting SLAM with Neural Radiance Prior

TL;DR

DenseSplat tackles the challenge of sparse-view SLAM by fusing Neural Radiance Field priors with Gaussian Splatting to densify and interpolate maps from sparse keyframes. The approach initializes Gaussian primitives via NeRF-derived surface transitions, renders with multi-scale Gaussians, and refines geometry through loop closure and BA within a submap framework for memory efficiency. Key contributions include geometry-aware primitive sampling, ray-guided pruning, and a real-time SLAM pipeline that delivers superior tracking and mapping on large-scale indoor scenes. This work advances practical dense SLAM for robotics and AR/VR by achieving high-fidelity reconstructions with sparse observations and real-time performance.

Abstract

Gaussian SLAM systems excel in real-time rendering and fine-grained reconstruction compared to NeRF-based systems. However, their reliance on extensive keyframes is impractical for deployment in real-world robotic systems, which typically operate under sparse-view conditions that can result in substantial holes in the map. To address these challenges, we introduce DenseSplat, the first SLAM system that effectively combines the advantages of NeRF and 3DGS. DenseSplat utilizes sparse keyframes and NeRF priors for initializing primitives that densely populate maps and seamlessly fill gaps. It also implements geometry-aware primitive sampling and pruning strategies to manage granularity and enhance rendering efficiency. Moreover, DenseSplat integrates loop closure and bundle adjustment, significantly enhancing frame-to-frame tracking accuracy. Extensive experiments on multiple large-scale datasets demonstrate that DenseSplat achieves superior performance in tracking and mapping compared to current state-of-the-art methods.

Figures (13)

• Figure 1: DenseSplat leverages NeRF priors into the Gaussian SLAM system, offering superior tracking, fine-grained mapping, and extraordinary real-time performance using sparse keyframes.
• Figure 2: Gaussian primitives initialized by direct backprojection from RGB-D streams suffer from several drawbacks. The left pixel image illustrates that closer objects occupy more pixels, whereas farther regions receive fewer rays. (a) Imbalanced point sampling occurs due to the uneven distribution of rays. (b) Erroneous projections arise from inaccurate depth estimates in distant regions. (c) Undersampled areas result from occlusions or insufficient views.
• Figure 3: DenseSplat comprises tracking and mapping modules. The tracking module computes camera poses by optimizing the NeRF model and streaming sparse keyframes to the mapping module. Gaussian primitives are produced via geometry-aware sampling, effectively capturing the scene geometry and seamlessly filling gaps. Enhanced by BA-induced map refinement and ray-guided Gaussian pruning strategies, DenseSplat delivers high-quality reconstructions at remarkable real-time speeds.
• Figure 4: Novel-view synthesis (NVS) comparison of DenseSplat and Gaussian-based baseline methods on the selected scenes of Replica apartment dataset straub2019replica. Our method demonstrates superior performance in geometric accuracy, hole filling, and fine-grained texture rendering. Crucially, DenseSplat utilizes a sparse keyframe (kf) interval of 20, offering a more efficient and practical setup compared to the dense keyframe lists employed by the baseline methods.
• Figure 5: NVS comparison on real-world ScanNet dataset dai2017scannet. DenseSplat shows superior geometry accuracy and hole filling.