FoundationSLAM: Unleashing the Power of Depth Foundation Models for End-to-End Dense Visual SLAM

TL;DR

FoundationSLAM tackles geometric inconsistencies in flow-based monocular dense SLAM by incorporating depth priors from foundation models into a fully differentiable pipeline. It introduces a Hybrid Flow Network, a Bi-Consistent Bundle Adjustment Layer, and a Reliability-Aware Refinement to enforce multi-view coherence and adapt refinement to unreliable regions, using residuals such as $L_{\text{flow}}$ and $L_{\text{geo}}$. The approach achieves state-of-the-art tracking and dense reconstruction on standard benchmarks and runs in real-time at 18 FPS, demonstrating strong generalization to varied environments. Overall, the work offers a practical, end-to-end solution for robust, dense SLAM with explicit geometric guidance and consistent optimization.

Abstract

We present FoundationSLAM, a learning-based monocular dense SLAM system that addresses the absence of geometric consistency in previous flow-based approaches for accurate and robust tracking and mapping. Our core idea is to bridge flow estimation with geometric reasoning by leveraging the guidance from foundation depth models. To this end, we first develop a Hybrid Flow Network that produces geometry-aware correspondences, enabling consistent depth and pose inference across diverse keyframes. To enforce global consistency, we propose a Bi-Consistent Bundle Adjustment Layer that jointly optimizes keyframe pose and depth under multi-view constraints. Furthermore, we introduce a Reliability-Aware Refinement mechanism that dynamically adapts the flow update process by distinguishing between reliable and uncertain regions, forming a closed feedback loop between matching and optimization. Extensive experiments demonstrate that FoundationSLAM achieves superior trajectory accuracy and dense reconstruction quality across multiple challenging datasets, while running in real-time at 18 FPS, demonstrating strong generalization to various scenarios and practical applicability of our method.

Figures (6)

• Figure 1: SLAM Performance Comparison. Radar plot shows normalized ATE (TUM, EuRoC, 7Scenes, ETH3D) and Chamfer distance (7Scenes, EuRoC). Our method (orange star) achieves optimal performance on any metrics.
• Figure 2: Method Overview. Given a pair of keyframes, we estimate dense optical flow using a hybrid network fusing geometry-aware features from a foundation depth model. Predicted flow is iteratively refined via Flow GRU, guided by context features and learned reliability masks. Refined flow drives a Bi-Consistent BA Layer jointly optimizing keyframe depth and pose using flow and geometry consistency residuals. Optimization feedback updates flow reliability in closed-loop manner. This process unrolls over multiple iterations to progressively improve accuracy and consistency.
• Figure 3: Qualitative Comparison on TNT Dataset. We show the qualitative results of our method and MASt3R-SLAM on the TNT tnt dataset, including the overall and detailed reconstruction, keyframe trajectories. Our method maintains significantly more keyframes while ensuring better geometric consistency, with less layering and artifacts.
• Figure 4: Qualitative Reconstruction Comparison. Comparison with SOTA baselines on EuRoC and 7Scenes.
• Figure 5: Impact of Bi-Consistent BA Layer. We visualize the depth maps of neighboring keyframes to reveal inconsistencies caused by unreliable optical flow in baseline systems. Our Bi-Consistent BA Layer significantly improves inter-frame geometric alignment.