Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MGSO: Monocular Real-time Photometric SLAM with Efficient 3D Gaussian Splatting

Yan Song Hu, Nicolas Abboud, Muhammad Qasim Ali, Adam Srebrnjak Yang, Imad Elhajj, Daniel Asmar, Yuhao Chen, John S. Zelek

TL;DR

MGSO tackles real-time dense monocular SLAM by coupling a photometric SLAM backbone with 3D Gaussian Splatting (3DGS). It initializes and guides 3DGS with a dense, structured RGB-derived point cloud, achieving high-quality, memory-efficient maps in real time on RGB-only input. Across Replica, EuRoC, and TUM-RGBD, MGSO outperforms contemporary 3DGS-based SLAM systems in map quality and memory usage while maintaining laptop-friendly performance. This work broadens the practicality of dense visual SLAM for robotics, AR/VR, and digital twin applications without depth sensors.

Abstract

Real-time SLAM with dense 3D mapping is computationally challenging, especially on resource-limited devices. The recent development of 3D Gaussian Splatting (3DGS) offers a promising approach for real-time dense 3D reconstruction. However, existing 3DGS-based SLAM systems struggle to balance hardware simplicity, speed, and map quality. Most systems excel in one or two of the aforementioned aspects but rarely achieve all. A key issue is the difficulty of initializing 3D Gaussians while concurrently conducting SLAM. To address these challenges, we present Monocular GSO (MGSO), a novel real-time SLAM system that integrates photometric SLAM with 3DGS. Photometric SLAM provides dense structured point clouds for 3DGS initialization, accelerating optimization and producing more efficient maps with fewer Gaussians. As a result, experiments show that our system generates reconstructions with a balance of quality, memory efficiency, and speed that outperforms the state-of-the-art. Furthermore, our system achieves all results using RGB inputs. We evaluate the Replica, TUM-RGBD, and EuRoC datasets against current live dense reconstruction systems. Not only do we surpass contemporary systems, but experiments also show that we maintain our performance on laptop hardware, making it a practical solution for robotics, A/R, and other real-time applications.

MGSO: Monocular Real-time Photometric SLAM with Efficient 3D Gaussian Splatting

TL;DR

MGSO tackles real-time dense monocular SLAM by coupling a photometric SLAM backbone with 3D Gaussian Splatting (3DGS). It initializes and guides 3DGS with a dense, structured RGB-derived point cloud, achieving high-quality, memory-efficient maps in real time on RGB-only input. Across Replica, EuRoC, and TUM-RGBD, MGSO outperforms contemporary 3DGS-based SLAM systems in map quality and memory usage while maintaining laptop-friendly performance. This work broadens the practicality of dense visual SLAM for robotics, AR/VR, and digital twin applications without depth sensors.

Abstract

Real-time SLAM with dense 3D mapping is computationally challenging, especially on resource-limited devices. The recent development of 3D Gaussian Splatting (3DGS) offers a promising approach for real-time dense 3D reconstruction. However, existing 3DGS-based SLAM systems struggle to balance hardware simplicity, speed, and map quality. Most systems excel in one or two of the aforementioned aspects but rarely achieve all. A key issue is the difficulty of initializing 3D Gaussians while concurrently conducting SLAM. To address these challenges, we present Monocular GSO (MGSO), a novel real-time SLAM system that integrates photometric SLAM with 3DGS. Photometric SLAM provides dense structured point clouds for 3DGS initialization, accelerating optimization and producing more efficient maps with fewer Gaussians. As a result, experiments show that our system generates reconstructions with a balance of quality, memory efficiency, and speed that outperforms the state-of-the-art. Furthermore, our system achieves all results using RGB inputs. We evaluate the Replica, TUM-RGBD, and EuRoC datasets against current live dense reconstruction systems. Not only do we surpass contemporary systems, but experiments also show that we maintain our performance on laptop hardware, making it a practical solution for robotics, A/R, and other real-time applications.
Paper Structure (13 sections, 2 equations, 6 figures, 8 tables)

This paper contains 13 sections, 2 equations, 6 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. SLAM module:
  5. Dense Reconstruction
  6. Experiments and Discussions
  7. Implementation and Setup
  8. Discussion
  9. Localization
  10. Reconstruction Quality
  11. Resource Efficiency and Real-Time Performance
  12. Ablations
  13. Conclusions

Figures (6)

  • Figure 1: Qualitative renders of the TUM-RGBD dataset sturm12iros with input point clouds. By initializing 3D Gaussian Splatting (3DGS) kerbl3Dgaussians with dense, structured point clouds, MGSO produces reconstructions that are memory-efficient and high quality.
  • Figure 2: Comparison of 3DGS point clouds from MGSO and Photo-SLAM on Replica room0. Top left: original frame; top right: Map from original 3DGS after 10,240 iterations with Gaussian size set to 0.1. Bottom: MGSO vs. Photo-SLAM point clouds.
  • Figure 3: Tracked points from ORBSLAM3 (left) compared to our system (right). Our system tracks much more points than ORBSLAM3, which results in denser point clouds outputs.
  • Figure 4: Original DSO Point cloud (left) compared to our system's point cloud (right). Our system has much more output points, especially in flat low-gradient regions.
  • Figure 5: Comparison of difficult novel view renders between MGSO (top) and Photo-SLAM (bottom). Captions describe how MGSO performs better.
  • ...and 1 more figures