SLAIM: Robust Dense Neural SLAM for Online Tracking and Mapping

TL;DR

SLAIM tackles the challenge of robust online dense SLAM with NeRF-based implicit maps by introducing a Gaussian Pyramid-based coarse-to-fine tracking and a KL regularizer over the ray termination distribution to constrain scene geometry. The approach preserves the original volume-density NeRF representation while enabling efficient tracking through image pyramids and improved depth consistency via the KL loss, coupled with both local and global bundle adjustment. Empirical results on Replica, ScanNet, and TUM-RGBD show state-of-the-art tracking performance and high-quality reconstructions, with favorable memory and runtime characteristics. The work advances practical dense RGB-D SLAM by marrying implicit scene representations with a robust, multi-scale optimization strategy that reduces local minima issues and promotes geometrically accurate mapping.”

Abstract

We present SLAIM - Simultaneous Localization and Implicit Mapping. We propose a novel coarse-to-fine tracking model tailored for Neural Radiance Field SLAM (NeRF-SLAM) to achieve state-of-the-art tracking performance. Notably, existing NeRF-SLAM systems consistently exhibit inferior tracking performance compared to traditional SLAM algorithms. NeRF-SLAM methods solve camera tracking via image alignment and photometric bundle-adjustment. Such optimization processes are difficult to optimize due to the narrow basin of attraction of the optimization loss in image space (local minima) and the lack of initial correspondences. We mitigate these limitations by implementing a Gaussian pyramid filter on top of NeRF, facilitating a coarse-to-fine tracking optimization strategy. Furthermore, NeRF systems encounter challenges in converging to the right geometry with limited input views. While prior approaches use a Signed-Distance Function (SDF)-based NeRF and directly supervise SDF values by approximating ground truth SDF through depth measurements, this often results in suboptimal geometry. In contrast, our method employs a volume density representation and introduces a novel KL regularizer on the ray termination distribution, constraining scene geometry to consist of empty space and opaque surfaces. Our solution implements both local and global bundle-adjustment to produce a robust (coarse-to-fine) and accurate (KL regularizer) SLAM solution. We conduct experiments on multiple datasets (ScanNet, TUM, Replica) showing state-of-the-art results in tracking and in reconstruction accuracy.

Figures (6)

• Figure 1: We present SLAIM, a robust dense neural RGB-D SLAM system that performs online tracking and mapping in real time. SLAIM implements a Gaussian Pyramid filter on top of NeRF to perform coarse-to-fine tracking and mapping. We also introduce a new target ray termination distribution that we use in a KL regularizer to supervise the network to converge towards the right geometry. SLAIM reaches state-of-the-art results in both tracking and 3D reconstruction accuracy.
• Figure 2: Overview of SLAIM. For a given input ray with center $\vec{o}$ and direction $\vec{r}$ we start by rendering its corresponding color pixel via ray tracing and volume rendering. For each sample $x_i$ along the ray we query the multiresolution feature grid to form an input embedding $y$. We then make successive calls to two shallow MLP networks to predict a density $\sigma$ and color $c$ for that sample. After reconstructing the image we apply a Gaussian Pyramid filter to perform coarse-to-fine tracking and mapping.
• Figure 3: Illustration of the custom ray termination distribution. (left-top) shows the estimation of the density response $\tilde{\sigma} \sim sech^2$ as a narrow bell-shape function. (left-bottom) compares the resulting ray termination distribution $\tilde{w}$ computed from $\tilde{\sigma}$ to the Gaussian distribution described in DS-NeRF deng2022depth. On the right, we display results from an experiment showcasing the ray termination distribution' shape under different conditions. (middle) compares mapping setups: one with multiple training views (over-constrained geometry) and another with only one view (under-constrained geometry). (right-top) shows multi-view training without KL regularization (right-middle) depicts single-view training without KL regularization, and (right-bottom) exhibits single-view training with our custom regularizer. We observe similarity between the ray termination distributions in the (right-top) and (right-bottom), supporting the use of our custom distribution.
• Figure 4: Reconstruction results on Replica straub2019replica. Our method can retrieve thin structures where other baselines tend to oversmooth the geometry.
• Figure 5: SLAIM qualitative results on the ScanNet dataset dai2017scannet. The ground truth camera trajectory is shown in green, and the estimated one in red. In comparison to other baselines johari2022eslamwang2023co our method achieves more accurate tracking results and produces finer 3D reconstruction (right column).