Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

M^3: Dense Matching Meets Multi-View Foundation Models for Monocular Gaussian Splatting SLAM

Kerui Ren, Guanghao Li, Changjian Jiang, Yingxiang Xu, Tao Lu, Linning Xu, Junting Dong, Jiangmiao Pang, Mulin Yu, Bo Dai

Abstract

Streaming reconstruction from uncalibrated monocular video remains challenging, as it requires both high-precision pose estimation and computationally efficient online refinement in dynamic environments. While coupling 3D foundation models with SLAM frameworks is a promising paradigm, a critical bottleneck persists: most multi-view foundation models estimate poses in a feed-forward manner, yielding pixel-level correspondences that lack the requisite precision for rigorous geometric optimization. To address this, we present M^3, which augments the Multi-view foundation model with a dedicated Matching head to facilitate fine-grained dense correspondences and integrates it into a robust Monocular Gaussian Splatting SLAM. M^3 further enhances tracking stability by incorporating dynamic area suppression and cross-inference intrinsic alignment. Extensive experiments on diverse indoor and outdoor benchmarks demonstrate state-of-the-art accuracy in both pose estimation and scene reconstruction. Notably, M^3 reduces ATE RMSE by 64.3% compared to VGGT-SLAM 2.0 and outperforms ARTDECO by 2.11 dB in PSNR on the ScanNet++ dataset.

M^3: Dense Matching Meets Multi-View Foundation Models for Monocular Gaussian Splatting SLAM

Abstract

Streaming reconstruction from uncalibrated monocular video remains challenging, as it requires both high-precision pose estimation and computationally efficient online refinement in dynamic environments. While coupling 3D foundation models with SLAM frameworks is a promising paradigm, a critical bottleneck persists: most multi-view foundation models estimate poses in a feed-forward manner, yielding pixel-level correspondences that lack the requisite precision for rigorous geometric optimization. To address this, we present M^3, which augments the Multi-view foundation model with a dedicated Matching head to facilitate fine-grained dense correspondences and integrates it into a robust Monocular Gaussian Splatting SLAM. M^3 further enhances tracking stability by incorporating dynamic area suppression and cross-inference intrinsic alignment. Extensive experiments on diverse indoor and outdoor benchmarks demonstrate state-of-the-art accuracy in both pose estimation and scene reconstruction. Notably, M^3 reduces ATE RMSE by 64.3% compared to VGGT-SLAM 2.0 and outperforms ARTDECO by 2.11 dB in PSNR on the ScanNet++ dataset.
Paper Structure (41 sections, 10 equations, 8 figures, 27 tables)

This paper contains 41 sections, 10 equations, 8 figures, 27 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Learning-based and Foundation Model-based SLAM
  4. 3D Scene Reconstruction
  5. Streaming Reconstruction
  6. Method
  7. Dense Matching through Foundation Model
  8. Model Architecture.
  9. Loss function and training.
  10. Dense matching for SLAM.
  11. Dynamic Region Estimation.
  12. M$^3$ Framework
  13. Sliding Window Management.
  14. Intrinsic Consistency.
  15. Tracking and Global Optimization.
  16. ...and 26 more sections

Figures (8)

  • Figure 1: We demonstrate the execution of our M$^3$ pipeline on a challenging manor sequence. Our approach delivers robust, high-precision pose estimation while achieving high-fidelity scene reconstruction from monocular video sequences. Please refer to our project page for more demos: https://city-super.github.io/M3/
  • Figure 2: The M$^3$ Pipeline. The framework consists of joint tracking and global optimization for pose estimation and a mapper for scene reconstruction. For monocular sequences, Pi3X processes retrieved historical keyframes and new frames in one inference to facilitate factor graph construction and keyframe selection. Following the Neural Gaussian and LOD architecture of ARTDECO artdeco, Gaussians are initialized via Laplacian norm and optimized jointly with camera poses.
  • Figure 3: Qualitative comparisons of rendering against on-the-fly reconstruction baselines across diverse datasets. M$^3$ preserves high-fidelity rendering details in challenging environments, particularly in regions highlighted by white rectangles.
  • Figure 4: Qualitative comparisons of rendering against feed-forward Gaussian Splatting methods. M$^3$ consistently preserves fine-grained visual details.
  • Figure 5: Effect of the Motion Map in dynamic environments. By detecting dynamic regions, the Motion Map enables moving objects to be excluded from the static reconstruction, thereby improving structural consistency in dynamic scenes.
  • ...and 3 more figures