Accurate Prior-centric Monocular Positioning with Offline LiDAR Fusion

TL;DR

This paper tackles centimeter-level monocular localization in GPS-denied environments by using a LiDAR-enhanced visual prior map built offline. The method performs geometry-aware visual mapping via a graph-based SfM with a geometry-aware bundle adjustment (GABA) that fuses visual observations with a LiDAR prior, and online monocular frame tracking with local map matching against the prior for real-time localization in $SE(3)$. A hierarchical localization strategy—relocalization and map tracking—enables reliable recovery from pose failures and robust long-term operation. Experiments on KITTI and campus datasets demonstrate centimeter-level accuracy and real-time performance on edge hardware, with deployment validated on an unmanned delivery platform, indicating a low-cost, scalable alternative to LiDAR-based localization.

Abstract

Unmanned vehicles usually rely on Global Positioning System (GPS) and Light Detection and Ranging (LiDAR) sensors to achieve high-precision localization results for navigation purpose. However, this combination with their associated costs and infrastructure demands, poses challenges for widespread adoption in mass-market applications. In this paper, we aim to use only a monocular camera to achieve comparable onboard localization performance by tracking deep-learning visual features on a LiDAR-enhanced visual prior map. Experiments show that the proposed algorithm can provide centimeter-level global positioning results with scale, which is effortlessly integrated and favorable for low-cost robot system deployment in real-world applications.

Figures (8)

• Figure 1: Image tracking in the visual feature map generated by the proposed method (top-right). The feature map can well capture both the visual detail and the precise geometry structure of the environment (top-left, top-mid, bottom).
• Figure 2: System pipeline of the proposed method.
• Figure 3: Illustration of the geometry-aware bundle adjustment.
• Figure 4: Reconstructed visual map w/o. and w/. structure factor. Introducing geometry constraints in visual mapping greatly improves the overall reconstruction quality.
• Figure 5: Illustration of the feature tracking process. The tracked features are colored based on depth, free keypoints are colored by white. After the map reloading step, more map points that are not tracked at the map matching step are kept in the next frame.