Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BEVMAPMATCH: Multimodal BEV Neural Map Matching for Robust Re-Localization of Autonomous Vehicles

Shounak Sural, Ragunathan Rajkumar

Abstract

Localization in GNSS-denied and GNSS-degraded environments is a challenge for the safe widespread deployment of autonomous vehicles. Such GNSS-challenged environments require alternative methods for robust localization. In this work, we propose BEVMapMatch, a framework for robust vehicle re-localization on a known map without the need for GNSS priors. BEVMapMatch uses a context-aware lidar+camera fusion method to generate multimodal Bird's Eye View (BEV) segmentations around the ego vehicle in both good and adverse weather conditions. Leveraging a search mechanism based on cross-attention, the generated BEV segmentation maps are then used for the retrieval of candidate map patches for map-matching purposes. Finally, BEVMapMatch uses the top retrieved candidate for finer alignment against the generated BEV segmentation, achieving accurate global localization without the need for GNSS. Multiple frames of generated BEV segmentation further improve localization accuracy. Extensive evaluations show that BEVMapMatch outperforms existing methods for re-localization in GNSS-denied and adverse environments, with a Recall@1m of 39.8%, being nearly twice as much as the best performing re-localization baseline. Our code and data will be made available at https://github.com/ssuralcmu/BEVMapMatch.git.

BEVMAPMATCH: Multimodal BEV Neural Map Matching for Robust Re-Localization of Autonomous Vehicles

Abstract

Localization in GNSS-denied and GNSS-degraded environments is a challenge for the safe widespread deployment of autonomous vehicles. Such GNSS-challenged environments require alternative methods for robust localization. In this work, we propose BEVMapMatch, a framework for robust vehicle re-localization on a known map without the need for GNSS priors. BEVMapMatch uses a context-aware lidar+camera fusion method to generate multimodal Bird's Eye View (BEV) segmentations around the ego vehicle in both good and adverse weather conditions. Leveraging a search mechanism based on cross-attention, the generated BEV segmentation maps are then used for the retrieval of candidate map patches for map-matching purposes. Finally, BEVMapMatch uses the top retrieved candidate for finer alignment against the generated BEV segmentation, achieving accurate global localization without the need for GNSS. Multiple frames of generated BEV segmentation further improve localization accuracy. Extensive evaluations show that BEVMapMatch outperforms existing methods for re-localization in GNSS-denied and adverse environments, with a Recall@1m of 39.8%, being nearly twice as much as the best performing re-localization baseline. Our code and data will be made available at https://github.com/ssuralcmu/BEVMapMatch.git.

Paper Structure

This paper contains 13 sections, 5 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. The BEVMapMatch Architecture
  4. Problem Definition
  5. Context-Aware BEV Representation Learning
  6. Coarse Map Localization via Patch Retrieval
  7. Fine-Grained Patch Alignment
  8. Evaluation
  9. Dataset and Experimental Settings
  10. Segmentation-guided patch retrieval performance
  11. Post-Retrieval Fine-grained Patch Alignment
  12. Operating Context-based Analysis
  13. Conclusion

Figures (5)

  • Figure 1: Sub-meter absolute re-localization with BEVMapMatch. Here, BEV segmentation maps are generated, followed by coarse matching that finds the most likely region, marked with a green square. This likely region is used to identify point-to-point correspondences against the segmented map. Finally, fine-grained pixel-level patch alignment yields accurate 3-DOF pose estimation (shown in pink).
  • Figure 2: Overview of the BEVMapMatch Architecture
  • Figure 3: Comparison of BEVMapMatch performance across choice of segmentation baselines and the ground truth.
  • Figure 4: Robustness of BEVMapMatch with random GNSS perturbation. BEVMapMatch-CAF stays stable over varying GNSS errors since it is retrieval-based and does not use GNSS priors.
  • Figure 5: Examples of localization at day and night-time with BEVMapMatch. Camera and lidar views shown (left) along with its localization output on a NuScenes base map (right). Within pink circles, red squares mark the predictions, blues mark ground truth and greens mark full overlap.