HERO-SLAM: Hybrid Enhanced Robust Optimization of Neural SLAM

TL;DR

This work tackles robustness gaps in neural implicit SLAM under sparse data and large viewpoint changes. It introduces HERO-SLAM, which fuses a multi-resolution implicit field with hybrid, warping-based cross-frame supervision to jointly enhance tracking and dense mapping. Key contributions include a multi-resolution implicit representation, TSDF-depth supervision, and a Hybrid Enhanced Robust Optimization that leverages homography warping, SSIM patch losses, and SuperPoint/LightGlue correspondences. Experimental results on Replica, ScanNet, and TUM demonstrate superior robustness and accuracy over state-of-the-art neural SLAM methods, including at lower image frequencies, with real-time capable performance. Overall, HERO-SLAM broadens the practical applicability of neural SLAM in real-world robotics and AR/VR tasks by improving stability and quality in challenging conditions.

Abstract

Simultaneous Localization and Mapping (SLAM) is a fundamental task in robotics, driving numerous applications such as autonomous driving and virtual reality. Recent progress on neural implicit SLAM has shown encouraging and impressive results. However, the robustness of neural SLAM, particularly in challenging or data-limited situations, remains an unresolved issue. This paper presents HERO-SLAM, a Hybrid Enhanced Robust Optimization method for neural SLAM, which combines the benefits of neural implicit field and feature-metric optimization. This hybrid method optimizes a multi-resolution implicit field and enhances robustness in challenging environments with sudden viewpoint changes or sparse data collection. Our comprehensive experimental results on benchmarking datasets validate the effectiveness of our hybrid approach, demonstrating its superior performance over existing implicit field-based methods in challenging scenarios. HERO-SLAM provides a new pathway to enhance the stability, performance, and applicability of neural SLAM in real-world scenarios. Code is available on the project page: https://hero-slam.github.io.

Figures (4)

• Figure 1: The visualization of mapping and tracking errors on Replica replica-dataset of challenging sparse inputs with large motion changes. This paper introduces a robust system for real-time dense 3D reconstruction, dubbed HERO-SLAM, which synergistically leverages the capabilities of neural implicit fields and feature-metric optimization, demonstrating exceptional resilience to large viewpoint changes and ensuring efficient runtime performance.
• Figure 2: The overview of HERO-SLAM. We use hybrid optimization to enhance the robustness of neural SLAM. Every newly captured frame would be aligned with the last frame for the camera pose estimation using feature-metric warping losses. The robustness and accuracy of tracking get improved, which in turn, facilitates the enhancement of mapping quality by optimizing the neural implicit field of multi-resolution feature encoding. The mapping module optimizes all keyframes from the keyframe database based on photometric reconstruction and depth supervision, following the volumetric rendering paradigm.
• Figure 3: Qualitative visualization of results among different approaches. Our reconstructions are smoother, more complete, and have fewer artifacts compared to other advanced methods on the ScanNet dataset scannet-dataset.
• Figure 4: A comparison of ATE RMSE trends as the number of images increases across different scenes.