Exploring Adversarial Robustness of LiDAR-Camera Fusion Model in Autonomous Driving

TL;DR

This work analyzes the adversarial robustness of LiDAR-camera fusion for 3D object detection in autonomous driving by introducing a physically constrained Hiding Attack that adds a small number of adversarial points atop a target vehicle using LiDAR data alone. The attack, restricted by per-ray point limits, discrete vertical angles, and narrow horizontal range, significantly degrades detection performance and raises safety concerns, with attack success increasing as the number of points grows and as the target car sits farther from the LiDAR or directly in front. Evaluations on the MVX-Net fusion model with the KITTI dataset reveal that even minimal LiDAR perturbations can render front-near cars undetectable, while closer or off-angle configurations modulate the ASR. The findings underscore the vulnerability of multi-sensor perception in autonomous driving and provide insights to enhance the robustness of fusion-based 3D object detection against physically realizable adversaries.

Abstract

Our study assesses the adversarial robustness of LiDAR-camera fusion models in 3D object detection. We introduce an attack technique that, by simply adding a limited number of physically constrained adversarial points above a car, can make the car undetectable by the fusion model. Experimental results reveal that even without changes to the image data channel, the fusion model can be deceived solely by manipulating the LiDAR data channel. This finding raises safety concerns in the field of autonomous driving. Further, we explore how the quantity of adversarial points, the distance between the front-near car and the LiDAR-equipped car, and various angular factors affect the attack success rate. We believe our research can contribute to the understanding of multi-sensor robustness, offering insights and guidance to enhance the safety of autonomous driving.

Figures (7)

• Figure 1: Adversarial points, when subjected to physical constraints, can deceive the LiDAR-camera fusion 3D detection model used in autonomous driving. By adding a limited number of these adversarial points above the central car, it becomes undetectable by the fusion model.
• Figure 2: Velodyne HDL-64E has 64 lasers, and their vertical angle range is -24.8° to 2°. 32 lasers in lower laser block separated by 1/2° vertical spacing and 32 lasers in upper laser block separated by 1/3° vertical spacing.
• Figure 3: Overview of the attack pipeline. The standard data processing flow of MVX-Net is indicated by solid black lines. To ensure the injectability of adversarial points into the victim LiDAR in the real world, these points are transformed based on a specific angle in the spherical coordinate system, using the LiDAR as the reference point. This additional data processing step is denoted by a solid red line. The flow of gradients is shown by dashed red lines. As the entire pipeline is differentiable, gradients can flow from the adversarial loss back to the adversarial points within the spherical coordinate system. Ultimately, cars with adversarial points situated above their roofs are rendered undetectable by the fusion model.
• Figure 4: Attack success rate of spoofing fusion model to make front-near cars undetected with different number of adversarial points. The ASR roughly increases with the number of added adversarial points.
• Figure 5: Recall-IoU curves with different IoU thresholds. The curve labeled with "clean" represents the performance of the fusion model MVX-Net sindagi2019mvx without interference from adversarial points that comply with physical constraints.