Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

P2U-SLAM: A Monocular Wide-FoV SLAM System Based on Point Uncertainty and Pose Uncertainty

Yufan Zhang, Kailun Yang, Ze Wang, Kaiwei Wang

TL;DR

P2U-SLAM tackles the challenges of monocular wide-FoV SLAM by explicitly modeling and updating the uncertainty of both 3D points and past poses. Point uncertainty ${Σ}_{3×3,p}$ is integrated into tracking, while pose uncertainty ${Σ}_{6×6,p}$ informs local BA, with all updates occurring after optimization steps such as local mapping, map merging, and loop closing. The approach leverages Taylor and Kannala-Brandt camera models to handle non-perspective wide-FoV projections and demonstrates superior robustness and accuracy on PALVIO and TUM-VI datasets, including map reuse scenarios, with real-time performance on CPU. This work advances wide-FoV SLAM by providing a principled uncertainty-aware optimization framework that mitigates weak constraints due to partial observations and re-observations over long-term operation.

Abstract

This paper presents P2U-SLAM, a visual Simultaneous Localization And Mapping (SLAM) system with a wide Field of View (FoV) camera, which utilizes pose uncertainty and point uncertainty. While the wide FoV enables considerable repetitive observations of historical map points for matching cross-view features, the data properties of the historical map points and the poses of historical keyframes have changed during the optimization process. The neglect of data property changes results in the lack of partial information matrices in optimization, increasing the risk of long-term positioning performance degradation. The purpose of our research is to mitigate the risks posed by wide-FoV visual input to the SLAM system. Based on the conditional probability model, this work reveals the definite impacts of the above data properties changes on the optimization process, concretizes these impacts as point uncertainty and pose uncertainty, and gives their specific mathematical form. P2U-SLAM embeds point uncertainty into the tracking module and pose uncertainty into the local mapping module respectively, and updates these uncertainties after each optimization operation including local mapping, map merging, and loop closing. We present an exhaustive evaluation on 27 sequences from two popular public datasets with wide-FoV visual input. P2U-SLAM shows excellent performance compared with other state-of-the-art methods. The source code will be made publicly available at https://github.com/BambValley/P2U-SLAM.

P2U-SLAM: A Monocular Wide-FoV SLAM System Based on Point Uncertainty and Pose Uncertainty

TL;DR

P2U-SLAM tackles the challenges of monocular wide-FoV SLAM by explicitly modeling and updating the uncertainty of both 3D points and past poses. Point uncertainty is integrated into tracking, while pose uncertainty informs local BA, with all updates occurring after optimization steps such as local mapping, map merging, and loop closing. The approach leverages Taylor and Kannala-Brandt camera models to handle non-perspective wide-FoV projections and demonstrates superior robustness and accuracy on PALVIO and TUM-VI datasets, including map reuse scenarios, with real-time performance on CPU. This work advances wide-FoV SLAM by providing a principled uncertainty-aware optimization framework that mitigates weak constraints due to partial observations and re-observations over long-term operation.

Abstract

This paper presents P2U-SLAM, a visual Simultaneous Localization And Mapping (SLAM) system with a wide Field of View (FoV) camera, which utilizes pose uncertainty and point uncertainty. While the wide FoV enables considerable repetitive observations of historical map points for matching cross-view features, the data properties of the historical map points and the poses of historical keyframes have changed during the optimization process. The neglect of data property changes results in the lack of partial information matrices in optimization, increasing the risk of long-term positioning performance degradation. The purpose of our research is to mitigate the risks posed by wide-FoV visual input to the SLAM system. Based on the conditional probability model, this work reveals the definite impacts of the above data properties changes on the optimization process, concretizes these impacts as point uncertainty and pose uncertainty, and gives their specific mathematical form. P2U-SLAM embeds point uncertainty into the tracking module and pose uncertainty into the local mapping module respectively, and updates these uncertainties after each optimization operation including local mapping, map merging, and loop closing. We present an exhaustive evaluation on 27 sequences from two popular public datasets with wide-FoV visual input. P2U-SLAM shows excellent performance compared with other state-of-the-art methods. The source code will be made publicly available at https://github.com/BambValley/P2U-SLAM.
Paper Structure (21 sections, 26 equations, 8 figures, 4 tables)

This paper contains 21 sections, 26 equations, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. VO/VIO and SLAM
  4. Wide-FoV SLAM
  5. Camera Model
  6. Taylor Camera Model
  7. Kannala-Brandt Fisheye Model
  8. Problem Statement
  9. Point Uncertainty
  10. Pose Uncertainty
  11. System Overview
  12. Initialization
  13. Tracking with Point Uncertainty
  14. Local BA with Pose Uncertainty
  15. Loop Closing
  16. ...and 6 more sections

Figures (8)

  • Figure 1: An example of the wide-FoV lens used in transportation systems. a) The aerial platform used for data collection in the PALVIO dataset wang2022lf, equipped with two wide-FoV Panoramic Annular Lens (PAL) cameras; b) The output trajectories of several SLAM/VIO algorithms on the ID01 sequence of the PALVIO dataset, where the blue and green trajectories are the results of the P2U-SLAM proposed in this paper. In addition, (w.) indicates running with the support of point uncertainty and pose uncertainty, and (w./o.) indicates running without that support.
  • Figure 2: Several kinds of wide-FoV images and their FoV distribution. a) The radial FoV distribution of the fisheye image is generally nonlinear; b) the reflective structure of PAL leads to the presence of a central blind area in PAL images, while it gains a larger optical design space to ensure a nearly linear radial FoV distribution; c) the panoramic image is a rectangle and its complete FoV is usually projected and stitched together with multiple cameras.
  • Figure 3: The pipeline of P2U-SLAM. P2U-SLAM mainly consists of initialization, tracking, local BA, and loop closing. Point uncertainty functions in the tracking module to suppress the noise from treating past map points as measurement results on the current frame's pose estimation. Pose uncertainty acts on the local BA module to suppress the noise from treating fixed keyframe poses as measurement results on other variables to estimate.
  • Figure 4: Illustrations of situations in which point and pose uncertainty are applied. (a) The map points are viewed as fixed when estimating the pose of a new frame of tracking, so the point uncertainty should be applied according to Eq. \ref{['Covariance_Point']} and Eq. \ref{['MLE_1C_ONLYPOSE_GUASS']}. (b) In the process of local BA, there are some fixed keyframes viewed as observation results that are not targets to estimate while still having a great influence on optimization. Similarly, the pose uncertainty of these fixed keyframes (with orange dotted boxes) should be processed as Eq. \ref{['MLE_1C_ONLYPOINT']} and Eq. \ref{['Covariance_POSE']}.
  • Figure 5: Examples of error analyses and top trajectories of different SLAM and VIO systems on the PALVIO dataset wang2022lf. The details of the a) Translation and Rotation Error subfigures illustrate that P2U-SLAM not only has a low average error but also exhibits a more concentrated and stable error distribution. In the b) Top Trajectory subfigures, the clear superiority of P2U-SLAM in overlap with the ground truth trajectory over VINS-Mono and LF-VIO demonstrates its advantage of low absolute trajectory error, while its higher trajectory smoothness compared to ORB-SLAM3 indicates the stability of its real-time tracking. This is attributed to the innovative integration of point- and pose uncertainty within the P2U-SLAM framework, which effectively mitigates the impact of weak feature correspondences and enhances the overall reliability of the system.
  • ...and 3 more figures