An Efficient Approach to Generate Safe Drivable Space by LiDAR-Camera-HDmap Fusion

TL;DR

This work tackles the challenge of robust drivable-space perception for autonomous vehicles under diverse and adverse conditions by fusing LiDAR, camera, and HD-map data. The authors introduce an adaptive, generalizable pipeline featuring (1) adaptive grid ground removal with curb detection guided by HD maps, (2) an adaptive DBSCAN clustering approach resilient to precipitation noise, (3) a cost-based LiDAR-camera frustum association with depth-informed metrics, and (4) a comprehensive drivable-space representation that respects vehicle dimensions and road rules. They validate the approach on the Waterloo WATonoBus dataset, showing significant reductions in miss rate and false alarms under heavy snow, light snow, and sunny conditions, as well as high IOU performance compared to baselines. The contributions—adaptive grounding, robust clustering, semantically aware fusion, and HD-map integration—offer a scalable, platform-agnostic solution with practical safety benefits for all-weather autonomous driving. These results underscore the potential of multi-sensor fusion and map priors to enhance reliability and safety in real-world AV operations.

Abstract

In this paper, we propose an accurate and robust perception module for Autonomous Vehicles (AVs) for drivable space extraction. Perception is crucial in autonomous driving, where many deep learning-based methods, while accurate on benchmark datasets, fail to generalize effectively, especially in diverse and unpredictable environments. Our work introduces a robust easy-to-generalize perception module that leverages LiDAR, camera, and HD map data fusion to deliver a safe and reliable drivable space in all weather conditions. We present an adaptive ground removal and curb detection method integrated with HD map data for enhanced obstacle detection reliability. Additionally, we propose an adaptive DBSCAN clustering algorithm optimized for precipitation noise, and a cost-effective LiDAR-camera frustum association that is resilient to calibration discrepancies. Our comprehensive drivable space representation incorporates all perception data, ensuring compatibility with vehicle dimensions and road regulations. This approach not only improves generalization and efficiency, but also significantly enhances safety in autonomous vehicle operations. Our approach is tested on a real dataset and its reliability is verified during the daily (including harsh snowy weather) operation of our autonomous shuttle, WATonoBus

Figures (4)

• Figure 1: Overview of the proposed method; multiple LiDAR point clouds, and in parallel, multiple camera images are received. Both undergo pre-processing, and then fused to come up with the reliable all-weather drivable space.
• Figure 2: Camera Projection
• Figure 3: Detection performance comparison under various setups.
• Figure 4: Detection performance in heavy snow conditions.