Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

360ORB-SLAM: A Visual SLAM System for Panoramic Images with Depth Completion Network

Yichen Chen, Yiqi Pan, Ruyu Liu, Haoyu Zhang, Guodao Zhang, Bo Sun, Jianhua Zhang

TL;DR

The paper tackles scale ambiguity and robustness limits in monocular SLAM by introducing 360ORB-SLAM, a panoramic, depth-completion–guided vSLAM system. It combines a panoramic triangulation module with a dense depth completion network (multi-scale confidence propagation and multimodal fusion) to produce dense depth maps that stabilize pose estimation and reduce scale drift. Key contributions include a panoramic triangulation pipeline using spherical epipolar constraints, a confidence-aware depth completion network, and an end-to-end dense-depth–assisted SLAM workflow, validated on a Carla-based panoramic dataset with improved scale accuracy and robustness over monocular baselines. The approach demonstrates strong performance in feature-rich panoramic scenes and offers practical benefits for AR/VR, Visual Assistance, and autonomous driving applications, with potential for real-world deployment and further optimization.

Abstract

To enhance the performance and effect of AR/VR applications and visual assistance and inspection systems, visual simultaneous localization and mapping (vSLAM) is a fundamental task in computer vision and robotics. However, traditional vSLAM systems are limited by the camera's narrow field-of-view, resulting in challenges such as sparse feature distribution and lack of dense depth information. To overcome these limitations, this paper proposes a 360ORB-SLAM system for panoramic images that combines with a depth completion network. The system extracts feature points from the panoramic image, utilizes a panoramic triangulation module to generate sparse depth information, and employs a depth completion network to obtain a dense panoramic depth map. Experimental results on our novel panoramic dataset constructed based on Carla demonstrate that the proposed method achieves superior scale accuracy compared to existing monocular SLAM methods and effectively addresses the challenges of feature association and scale ambiguity. The integration of the depth completion network enhances system stability and mitigates the impact of dynamic elements on SLAM performance.

360ORB-SLAM: A Visual SLAM System for Panoramic Images with Depth Completion Network

TL;DR

The paper tackles scale ambiguity and robustness limits in monocular SLAM by introducing 360ORB-SLAM, a panoramic, depth-completion–guided vSLAM system. It combines a panoramic triangulation module with a dense depth completion network (multi-scale confidence propagation and multimodal fusion) to produce dense depth maps that stabilize pose estimation and reduce scale drift. Key contributions include a panoramic triangulation pipeline using spherical epipolar constraints, a confidence-aware depth completion network, and an end-to-end dense-depth–assisted SLAM workflow, validated on a Carla-based panoramic dataset with improved scale accuracy and robustness over monocular baselines. The approach demonstrates strong performance in feature-rich panoramic scenes and offers practical benefits for AR/VR, Visual Assistance, and autonomous driving applications, with potential for real-world deployment and further optimization.

Abstract

To enhance the performance and effect of AR/VR applications and visual assistance and inspection systems, visual simultaneous localization and mapping (vSLAM) is a fundamental task in computer vision and robotics. However, traditional vSLAM systems are limited by the camera's narrow field-of-view, resulting in challenges such as sparse feature distribution and lack of dense depth information. To overcome these limitations, this paper proposes a 360ORB-SLAM system for panoramic images that combines with a depth completion network. The system extracts feature points from the panoramic image, utilizes a panoramic triangulation module to generate sparse depth information, and employs a depth completion network to obtain a dense panoramic depth map. Experimental results on our novel panoramic dataset constructed based on Carla demonstrate that the proposed method achieves superior scale accuracy compared to existing monocular SLAM methods and effectively addresses the challenges of feature association and scale ambiguity. The integration of the depth completion network enhances system stability and mitigates the impact of dynamic elements on SLAM performance.
Paper Structure (17 sections, 12 equations, 9 figures, 3 tables)

This paper contains 17 sections, 12 equations, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Camera Model
  3. The Proposed Method
  4. Panoramic Triangulation Module
  5. Sparse Depth Generation Module
  6. Dense Depth Completion Module
  7. Multi-scale Confidence Propagation Network
  8. Depth completion network based on multimodal fusion
  9. Dense Depth Application Module
  10. Experiments
  11. Dataset
  12. Experimental Setup
  13. Results and Analysis
  14. Feature Detection and Matching
  15. Depth Completion
  16. ...and 2 more sections

Figures (9)

  • Figure 1: Coordinate systems
  • Figure 2: Triangulation based on a panoramic camera model
  • Figure 3: Panoramic projection flow chart
  • Figure 4: Deep completion network framework diagram
  • Figure 5: Comparison of monocular and panoramic feature detection
  • ...and 4 more figures