Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HS-SLAM: Hybrid Representation with Structural Supervision for Improved Dense SLAM

Ziren Gong, Fabio Tosi, Youmin Zhang, Stefano Mattoccia, Matteo Poggi

TL;DR

HS-SLAM addresses key challenges in dense SLAM by integrating a hybrid scene representation that fuses hash grids, tri-planes, and one-blob encodings, enabling complete and textured scene reconstructions. It introduces non-local patch-based structural supervision and an active global bundle adjustment to maintain global consistency during significant motion or forgetting. Across Replica, ScanNet, and TUM RGB-D, HS-SLAM outperforms state-of-the-art NeRF-centric baselines in both tracking accuracy and reconstruction quality while remaining computationally efficient for robotics. The approach offers a practical balance between accuracy and speed, with potential extensions to submaps and loop closure for larger-scale environments.

Abstract

NeRF-based SLAM has recently achieved promising results in tracking and reconstruction. However, existing methods face challenges in providing sufficient scene representation, capturing structural information, and maintaining global consistency in scenes emerging significant movement or being forgotten. To this end, we present HS-SLAM to tackle these problems. To enhance scene representation capacity, we propose a hybrid encoding network that combines the complementary strengths of hash-grid, tri-planes, and one-blob, improving the completeness and smoothness of reconstruction. Additionally, we introduce structural supervision by sampling patches of non-local pixels rather than individual rays to better capture the scene structure. To ensure global consistency, we implement an active global bundle adjustment (BA) to eliminate camera drifts and mitigate accumulative errors. Experimental results demonstrate that HS-SLAM outperforms the baselines in tracking and reconstruction accuracy while maintaining the efficiency required for robotics.

HS-SLAM: Hybrid Representation with Structural Supervision for Improved Dense SLAM

TL;DR

HS-SLAM addresses key challenges in dense SLAM by integrating a hybrid scene representation that fuses hash grids, tri-planes, and one-blob encodings, enabling complete and textured scene reconstructions. It introduces non-local patch-based structural supervision and an active global bundle adjustment to maintain global consistency during significant motion or forgetting. Across Replica, ScanNet, and TUM RGB-D, HS-SLAM outperforms state-of-the-art NeRF-centric baselines in both tracking accuracy and reconstruction quality while remaining computationally efficient for robotics. The approach offers a practical balance between accuracy and speed, with potential extensions to submaps and loop closure for larger-scale environments.

Abstract

NeRF-based SLAM has recently achieved promising results in tracking and reconstruction. However, existing methods face challenges in providing sufficient scene representation, capturing structural information, and maintaining global consistency in scenes emerging significant movement or being forgotten. To this end, we present HS-SLAM to tackle these problems. To enhance scene representation capacity, we propose a hybrid encoding network that combines the complementary strengths of hash-grid, tri-planes, and one-blob, improving the completeness and smoothness of reconstruction. Additionally, we introduce structural supervision by sampling patches of non-local pixels rather than individual rays to better capture the scene structure. To ensure global consistency, we implement an active global bundle adjustment (BA) to eliminate camera drifts and mitigate accumulative errors. Experimental results demonstrate that HS-SLAM outperforms the baselines in tracking and reconstruction accuracy while maintaining the efficiency required for robotics.

Paper Structure

This paper contains 16 sections, 11 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional SLAM Frameworks
  4. NeRF-centric Frameworks
  5. 3DGS-centric Framworks
  6. Method
  7. Hybrid Scene Representation
  8. SDF-based Rendering and Structural Supervision
  9. Active Global Bundle Adjustment
  10. Experiments
  11. Experimental setup
  12. Results on Replica
  13. Ablation Studies
  14. Results on Other Datasets
  15. Performance Analysis
  16. ...and 1 more sections

Figures (3)

  • Figure 2: The overview of HS-SLAM. 1) Tracking thread: The proposed global BA actively optimizes our system taking ray sampling from historical observations. 2) Mapping thread: The system optimizes hybrid encodings by performing bundle adjustment. 3) Hybrid scene representation: We carefully designed the encoders composed of hash grid, tri-planes, and one-blob to improve completeness and accuracy. Meanwhile, the proposed structural supervision (patch rendering loss) allows the network to capture global scene structures.
  • Figure 3: Differences between the pixel-wise and structural supervision. This figure illustrates the key process of the pixel-wise and structural supervision. Our structural supervision captures non-local information from scenes.
  • Figure 4: Qualitative reconstruction results on Replica. For each scene, we provide detailed drawings, highlighting complex regions to emphasize the textures and completeness of our reconstructed scenes. HS-SLAM demonstrates superior scene representation capabilities, effectively capturing intricate textures (e.g., flowers and nightstands) and achieving higher completeness in challenging areas (e.g., chairs and walls).