Monocular visual simultaneous localization and mapping: (r)evolution from geometry to deep learning-based pipelines

TL;DR

Monocular visual SLAM operates over states in $SE(3)$ and maps that can be sparse, dense, or semantic. The paper surveys geometry-based versus learning-based SLAM, detailing front-end and back-end modules and exploring end-to-end DL pipelines. It presents environment-specific challenges and experimental results showing geometry-based methods maintain accuracy in data-scarce or motion-diverse regimes, while DL methods provide resilience in degraded imaging but struggle with generalization due to limited training data. The work outlines open problems and directions toward integrating geometric SLAM with learning-based components for robust, scalable monocular navigation.

Abstract

With the rise of deep learning, there is a fundamental change in visual SLAM algorithms toward developing different modules trained as end-to-end pipelines. However, regardless of the implementation domain, visual SLAM's performance is subject to diverse environmental challenges, such as dynamic elements in outdoor environments, harsh imaging conditions in underwater environments, or blurriness in high-speed setups. These environmental challenges need to be identified to study the real-world viability of SLAM implementations. Motivated by the aforementioned challenges, this paper surveys the current state of visual SLAM algorithms according to the two main frameworks: geometry-based and learning-based SLAM. First, we introduce a general formulation of the SLAM pipeline that includes most of the implementations in the literature. Second, those implementations are classified and surveyed for geometry and learning-based SLAM. After that, environment-specific challenges are formulated to enable experimental evaluation of the resilience of different visual SLAM classes to varying imaging conditions. We address two significant issues in surveying visual SLAM, providing (1) a consistent classification of visual SLAM pipelines and (2) a robust evaluation of their performance under different deployment conditions. Finally, we give our take on future opportunities for visual SLAM implementations.

Figures (11)

• Figure 1: Standard modules for SLAM pipeline implementations as outlined in the paper. The front-end is where the raw data is extracted from the sensors and abstracted into a model. The back-end infers from the front-end data and optimizes the estimates. The main modules in the front-end are tracking, loop detection, and relocalization. Tracking can be either direct or indirect. The back-end can be classified into filter and optimization-based techniques.
• Figure 2: Direct SLAM bases the tracking on optimizing the photometric error. Given two image frames $I_{k-1}$ and $I_k$, the 3D point $P_i$ is projected into $p^{k-1}_i$ and $p^k_{i}$, respectively. The photometric error is the intensity difference between $p^{k-1}_i$ and $p^k_{i}$, obtained by projecting $I_k$ onto $I_{k-1}$.
• Figure 3: Sparse SLAM optimizing the reprojection error for tracking. Two matched points $p^{k-1}_i$ and $p^{k}_i$ are triangulated onto the estimated 3D point $\hat{P}_i$, which is then reprojected as $\hat{p}^{k-1}_i$ and $\hat{p}^{k}_i$. The error is the difference between reprojected and original position of the points.
• Figure 4: The BoW framework represents the image by extracting and clustering the descriptor vectors. Each cluster represents a visual word. The histogram of words comprises the BoW vector that describes the image's appearance.
• Figure 5: According to its back-end, SLAM can be classified as (a) filter-based, and (b) keyframe-based. Filter-based approaches marginalise all camera poses onto the last pose $C_k$, which is stored in the state vector with all landmarks $p_k$. Keyframe-based approaches only process data from the keyframes $K_i$, generating a sparse graph with the camera poses $C_k$, the landmarks $p_k$, and the map points $P_i$.