Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Learning-Powered Visual SLAM Aimed at Assisting Visually Impaired Navigation

Marziyeh Bamdad, Hans-Peter Hutter, Alireza Darvishy

TL;DR

This work targets robust visual SLAM for navigation aids for visually impaired users under challenging conditions such as low texture and rapid lighting changes. It introduces SELM-SLAM3, a front-end enhancement of ORB-SLAM3 that substitutes traditional feature extraction and matching with SuperPoint and LightGlue, respectively. Across diverse datasets (TUM RGB-D, ICL-NUIM, TartanAir), SELM-SLAM3 achieves substantial improvements in pose accuracy—averaging an 87.84% gain over ORB-SLAM3 and 36.77% over state-of-the-art RGB-D SLAM systems—demonstrating greater robustness in adverse environments. The approach advances practical assistive navigation by delivering more reliable localization and denser maps, while highlighting the need for tailored, real-world datasets and real-time optimization for deployment.

Abstract

Despite advancements in SLAM technologies, robust operation under challenging conditions such as low-texture, motion-blur, or challenging lighting remains an open challenge. Such conditions are common in applications such as assistive navigation for the visually impaired. These challenges undermine localization accuracy and tracking stability, reducing navigation reliability and safety. To overcome these limitations, we present SELM-SLAM3, a deep learning-enhanced visual SLAM framework that integrates SuperPoint and LightGlue for robust feature extraction and matching. We evaluated our framework using TUM RGB-D, ICL-NUIM, and TartanAir datasets, which feature diverse and challenging scenarios. SELM-SLAM3 outperforms conventional ORB-SLAM3 by an average of 87.84% and exceeds state-of-the-art RGB-D SLAM systems by 36.77%. Our framework demonstrates enhanced performance under challenging conditions, such as low-texture scenes and fast motion, providing a reliable platform for developing navigation aids for the visually impaired.

Deep Learning-Powered Visual SLAM Aimed at Assisting Visually Impaired Navigation

TL;DR

This work targets robust visual SLAM for navigation aids for visually impaired users under challenging conditions such as low texture and rapid lighting changes. It introduces SELM-SLAM3, a front-end enhancement of ORB-SLAM3 that substitutes traditional feature extraction and matching with SuperPoint and LightGlue, respectively. Across diverse datasets (TUM RGB-D, ICL-NUIM, TartanAir), SELM-SLAM3 achieves substantial improvements in pose accuracy—averaging an 87.84% gain over ORB-SLAM3 and 36.77% over state-of-the-art RGB-D SLAM systems—demonstrating greater robustness in adverse environments. The approach advances practical assistive navigation by delivering more reliable localization and denser maps, while highlighting the need for tailored, real-world datasets and real-time optimization for deployment.

Abstract

Despite advancements in SLAM technologies, robust operation under challenging conditions such as low-texture, motion-blur, or challenging lighting remains an open challenge. Such conditions are common in applications such as assistive navigation for the visually impaired. These challenges undermine localization accuracy and tracking stability, reducing navigation reliability and safety. To overcome these limitations, we present SELM-SLAM3, a deep learning-enhanced visual SLAM framework that integrates SuperPoint and LightGlue for robust feature extraction and matching. We evaluated our framework using TUM RGB-D, ICL-NUIM, and TartanAir datasets, which feature diverse and challenging scenarios. SELM-SLAM3 outperforms conventional ORB-SLAM3 by an average of 87.84% and exceeds state-of-the-art RGB-D SLAM systems by 36.77%. Our framework demonstrates enhanced performance under challenging conditions, such as low-texture scenes and fast motion, providing a reliable platform for developing navigation aids for the visually impaired.
Paper Structure (13 sections, 7 figures, 4 tables)

This paper contains 13 sections, 7 figures, 4 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. SLAM-Based Navigation Solutions for BVI
  4. From Classical SLAM to Deep Learning Techniques
  5. IMPLEMENTATION
  6. Feature Extraction
  7. Feature Matching
  8. Tracking
  9. EXPEREMENTS AND RESULTS
  10. TUM RGB-D dataset evaluation
  11. ICL-NUIM dataset evaluation
  12. TartanAir dataset evaluation
  13. CONCLUSIONS

Figures (7)

  • Figure 1: SELM-SLAM3 front-end, highlighting the integration of SuperPoint and LightGlue. The green boxes indicate the modified modules in ORB-SLAM3; the blue box represents the new module implemented in SELM-SLAM3; and the gray boxes depict the original modules of ORB-SLAM3 without modifications.
  • Figure 2: Comparison of ATE (m) on TUM RGB-D sequences illustrates the enhanced accuracy achieved by SELM-SLAM3 and the improvement over ORB-SLAM3.
  • Figure 3: Sample of estimated trajectory and ground-truth.
  • Figure 4: Comparison of ATE on ICL-NUIM dataset.
  • Figure 5: Challenging scene in lr-kt3
  • ...and 2 more figures