Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RaLF: Flow-based Global and Metric Radar Localization in LiDAR Maps

Abhijeet Nayak, Daniele Cattaneo, Abhinav Valada

TL;DR

This paper proposes RaLF, a novel deep neural network-based approach for localizing radar scans in a LiDAR map of the environment, by jointly learning to address both place recognition and metric localization.

Abstract

Localization is paramount for autonomous robots. While camera and LiDAR-based approaches have been extensively investigated, they are affected by adverse illumination and weather conditions. Therefore, radar sensors have recently gained attention due to their intrinsic robustness to such conditions. In this paper, we propose RaLF, a novel deep neural network-based approach for localizing radar scans in a LiDAR map of the environment, by jointly learning to address both place recognition and metric localization. RaLF is composed of radar and LiDAR feature encoders, a place recognition head that generates global descriptors, and a metric localization head that predicts the 3-DoF transformation between the radar scan and the map. We tackle the place recognition task by learning a shared embedding space between the two modalities via cross-modal metric learning. Additionally, we perform metric localization by predicting pixel-level flow vectors that align the query radar scan with the LiDAR map. We extensively evaluate our approach on multiple real-world driving datasets and show that RaLF achieves state-of-the-art performance for both place recognition and metric localization. Moreover, we demonstrate that our approach can effectively generalize to different cities and sensor setups than the ones used during training. We make the code and trained models publicly available at http://ralf.cs.uni-freiburg.de.

RaLF: Flow-based Global and Metric Radar Localization in LiDAR Maps

TL;DR

This paper proposes RaLF, a novel deep neural network-based approach for localizing radar scans in a LiDAR map of the environment, by jointly learning to address both place recognition and metric localization.

Abstract

Localization is paramount for autonomous robots. While camera and LiDAR-based approaches have been extensively investigated, they are affected by adverse illumination and weather conditions. Therefore, radar sensors have recently gained attention due to their intrinsic robustness to such conditions. In this paper, we propose RaLF, a novel deep neural network-based approach for localizing radar scans in a LiDAR map of the environment, by jointly learning to address both place recognition and metric localization. RaLF is composed of radar and LiDAR feature encoders, a place recognition head that generates global descriptors, and a metric localization head that predicts the 3-DoF transformation between the radar scan and the map. We tackle the place recognition task by learning a shared embedding space between the two modalities via cross-modal metric learning. Additionally, we perform metric localization by predicting pixel-level flow vectors that align the query radar scan with the LiDAR map. We extensively evaluate our approach on multiple real-world driving datasets and show that RaLF achieves state-of-the-art performance for both place recognition and metric localization. Moreover, we demonstrate that our approach can effectively generalize to different cities and sensor setups than the ones used during training. We make the code and trained models publicly available at http://ralf.cs.uni-freiburg.de.
Paper Structure (14 sections, 10 equations, 5 figures, 5 tables)

This paper contains 14 sections, 10 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Technical Approach
  4. Feature Extraction
  5. Place Recognition Head
  6. Metric Localization Head
  7. Inference
  8. Experimental Evaluation
  9. Datasets
  10. Training Details
  11. Place Recognition
  12. Metric Localization
  13. Ablation Studies
  14. Conclusion

Figures (5)

  • Figure 1: Our proposed RaLF localizes a radar scan within a LiDAR map both at a global (place recognition) and metric scale.
  • Figure 2: Overview of RaLF for joint place recognition and metric localization of radar scans in a LiDAR map. It consists of feature encoders, a place recognition head to extract global descriptors, and a metric localization head to estimate the 3-DoF pose of the query radar scan within the LiDAR map.
  • Figure 3: Train-test split of the three datasets used in our experiments. The blue and red trajectories represent the train and test splits, respectively.
  • Figure 4: Recall@k ($3m$) at different values of k.
  • Figure 5: Qualitative results of radar scans (grayscale) aligned with the LiDAR submaps (green) using our proposed method.