Get It For Free: Radar Segmentation without Expert Labels and Its Application in Odometry and Localization

Abstract

This paper presents a novel weakly supervised semantic segmentation method for radar segmentation, where the existing LiDAR semantic segmentation models are employed to generate semantic labels, which then serve as supervision signals for training a radar semantic segmentation model. The obtained radar semantic segmentation model outperforms LiDAR-based models, providing more consistent and robust segmentation under all-weather conditions, particularly in the snow, rain and fog. To mitigate potential errors in LiDAR semantic labels, we design a dedicated refinement scheme that corrects erroneous labels based on structural features and distribution patterns. The semantic information generated by our radar segmentation model is used in two downstream tasks, achieving significant performance improvements. In large-scale radar-based localization using OpenStreetMap, it leads to localization error reduction by 20.55\% over prior methods. For the odometry task, it improves translation accuracy by 16.4\% compared to the second-best method, securing the first place in the radar odometry competition at the Radar in Robotics workshop of ICRA 2024, Japan

Figures (10)

• Figure 1: Comparison of the information obtained by the three sensors under three different weather conditions. In rainy and snowy weather, camera images exhibit significant blurring, LiDAR data is heavily obstructed, while the radar remains stable. Meanwhile, our network continues to perform reliable semantic segmentation.
• Figure 2: System overview. The system employs another model to generate labels for the LiDAR point cloud. Following label refinement and LiDAR data filtering, the data is projected to create a supervisory signal with specific semantic information. This enables semantic segmentation of radar for downstream tasks.
• Figure 3: The red box indicates the building captured in the left mono camera image. While the radar can only detect information from the level with windows, the LiDAR can capture both the same level as the radar and the section between the two window levels.
• Figure 4: Comparison before and after the FOV Filter. Colorful rectangles indicate that after the filter, a significant amount of radar-invisible data has been removed from the supervisory LiDAR signal, resulting in a noticeable reduction in false positives in the network's output. Red circles in yellow rectangles highlight that the FOV Filter effectively removes residual ground point clouds.
• Figure 5: Comparison before and after the label refinement. The red rectangle in the camera view contains a building that has been mistakenly segmented as vegetation. Since the matrix formed by its point cloud coordinates produces two significantly larger singular values (compared to the third), we are able to correct the label to the proper one.