Pointing the Way: Refining Radar-Lidar Localization Using Learned ICP Weights

TL;DR

The paper tackles radar-lidar localization under adverse weather by refining ICP with a learned weight mask that filters radar points before alignment to lidar maps. A U-Net-based weight generator produces per-pixel weights from radar scans, while a stand-alone differentiable ICP (dICP) framework enables end-to-end training and backpropagation through the alignment process in $SE(2)$ for 2D pose. A supervisory map mask and BCE loss stabilize training, and the method demonstrates improved RMSE, convergence rates, and bias reduction on the Boreas dataset compared to unweighted baselines. The work contributes an open-source differentiable ICP library and a practical approach for robust, all-weather radar-lidar localization with potential impact on autonomous driving safety and robustness.

Abstract

This paper presents a novel deep-learning-based approach to improve localizing radar measurements against lidar maps. This radar-lidar localization leverages the benefits of both sensors; radar is resilient against adverse weather, while lidar produces high-quality maps in clear conditions. However, owing in part to the unique artefacts present in radar measurements, radar-lidar localization has struggled to achieve comparable performance to lidar-lidar systems, preventing it from being viable for autonomous driving. This work builds on ICP-based radar-lidar localization by including a learned preprocessing step that weights radar points based on high-level scan information. To train the weight-generating network, we present a novel, stand-alone, open-source differentiable ICP library. The learned weights facilitate ICP by filtering out harmful radar points related to artefacts, noise, and even vehicles on the road. Combining an analytical approach with a learned weight reduces overall localization errors and improves convergence in radar-lidar ICP results run on real-world autonomous driving data. Our code base is publicly available to facilitate reproducibility and extensions.

Figures (4)

• Figure 1: A satellite image of the environment, overlaid with the radar pointcloud weighted using our method (red, top), full extracted radar pointcloud (red, bottom), and lidar map (blue). The weight associated with each point in the weighted pointcloud corresponds to the intensity in colour. Our method picks out most of the important geometric features while ignoring noisy points and radar artefacts, thus facilitating the task of ICP.
• Figure 2: The pipeline used to train the weight mask network. A radar scan is fed to a U-Net network that produces a weight mask. In parallel, the scan is used to extract a pointcloud using the BFAR detector. The pointcloud then indexes weights from the weight mask. The points and corresponding weights are used in a weighted differentiable ICP algorithm (dICP) to localize against the reference map pointcloud. The training loss is formed from the error in the dICP result and a binary cross-entropy (BCE) term computed between the weight mask and a supervisory map mask. The network is trained by backpropagating the loss through dICP and the weight extraction.
• Figure 3: Three examples of the output from the pipeline in different scenarios. The mask highlights nearby structured areas when they exist (suburban road), focuses on reliable roadside signs and posts (intersection), all while ignoring cars (intersetion) and unique radar artefacts (narrow road).
• Figure 4: Box plot showing the distribution of errors between the baseline and our method in longitudinal translation, lateral translation, and heading. The results are presented for a changing noise scale, which is a factor applied to the boundaries of a uniform distribution from which initial guesses are sampled. Our method has a tighter error distribution in all components and across different initialization noise scales compared to the baseline.