S3-SLAM: Sparse Tri-plane Encoding for Neural Implicit SLAM

TL;DR

This work tackles the parameter-performance bottleneck in neural implicit SLAM by introducing Sparse Tri-plane Encoding, a compact representation that uses three orthogonal 2D hash-grid planes to encode geometry and appearance. By combining multi-resolution sparse tri-planes with lightweight decoders and a depth-guided rendering framework, S3-SLAM achieves high-resolution (up to 512) reconstructions with only about $2$–$4\%$ of the parameters of standard tri-planes, enabling rapid tracking and mapping. A hierarchical bundle adjustment (HBA) enforces global geometric consistency while preserving fine-grained local appearance, yielding competitive tracking and high-fidelity reconstructions across Replica, ScanNet, and TUM RGB-D datasets. The approach demonstrates significant memory efficiency and fast convergence, with potential impact on real-time neural implicit SLAM systems and large-scale scene understanding, and code will be released for public use.

Abstract

With the emergence of Neural Radiance Fields (NeRF), neural implicit representations have gained widespread applications across various domains, including simultaneous localization and mapping. However, current neural implicit SLAM faces a challenging trade-off problem between performance and the number of parameters. To address this problem, we propose sparse tri-plane encoding, which efficiently achieves scene reconstruction at resolutions up to 512 using only 2~4% of the commonly used tri-plane parameters (reduced from 100MB to 2~4MB). On this basis, we design S3-SLAM to achieve rapid and high-quality tracking and mapping through sparsifying plane parameters and integrating orthogonal features of tri-plane. Furthermore, we develop hierarchical bundle adjustment to achieve globally consistent geometric structures and reconstruct high-resolution appearance. Experimental results demonstrate that our approach achieves competitive tracking and scene reconstruction with minimal parameters on three datasets. Source code will soon be available.

Figures (16)

• Figure 1: We propose S3-SLAM, a neural implicit SLAM that applies our designed sparse tri-plane encoding. S3-SLAM can estimate accurate camera poses and reconstruct high-fidelity scenes with minimal memory footprint. The parameter count of our sparse tri-plane encoding is only 2$\sim$4% of the tri-planegao2022get3d (reduced from 100MB to 2$\sim$4MB).
• Figure 2: In our design, the hash-gridmuller2022instantngp (left) generates smoother and more coherent surface geometry than permutohedral lattice-gridrosu2023permutosdf (right).
• Figure 3: Our approach consists of neural implicit representation, tracking, and mapping. Given RGB-D input stream, S3-SLAM utilizes predicted poses and ray sampling to obtain 3D sampled points. Our sparse tri-plane encoding hashes multi-resolution plane features of sampled points to represent geometry and appearance compactly and efficiently. We represent $H_{xy}$, $H_{xz}$, and $H_{yz}$ of \ref{['eq:eq2']} using Hash(xy), Hash(xz), and Hash(yz). Our decoder decodes these encodings into SDF and color. During tracking and mapping, we adopt re-rendering loss of \ref{['eq:loss']} to update parameters of S3-SLAM.
• Figure 4: Qualitative comparison of the Replicastraub2019replica dataset shows S3-SLAM can reconstruct complete geometry of unknown viewpoints while reconstructing fine-level appearance. S3-SLAM employs sparse tri-plane encoding, achieving higher-resolution scene reconstruction.
• Figure 5: Qualitative comparison of geometric results on the ScanNetdai2017scannet dataset demonstrates our method can produce complete and coherent geometry. Compared to Co-SLAMwang2023coslam, S3-SLAM can reconstruct higher-resolution geometry, especially regarding surface contours. In contrast to Point-SLAMsandstrom2023pointslam, our method can generate more accurate and complete geometry.