A New Adversarial Perspective for LiDAR-based 3D Object Detection

TL;DR

Addressing safety challenges in LiDAR-based 3D object detection for autonomous driving, the paper proposes a real-world adversarial framework that uses random objects (water mist and smoke) to degrade perception. It introduces PCS-GAN to generate realistic point-cloud sequences of random objects and a range-image LiDAR renderer to simulate scanning occlusions, paired with a fusion pipeline to integrate perturbations onto target vehicles. The approach is validated on KITTI and nuScenes, showing high attack success rates across multiple detectors, and the authors release the ROLiD dataset to spur further research. The work also demonstrates that data augmentation with water-mist can improve robustness, highlighting both a novel threat and a potential defensive direction for future perception systems.

Abstract

Autonomous vehicles (AVs) rely on LiDAR sensors for environmental perception and decision-making in driving scenarios. However, ensuring the safety and reliability of AVs in complex environments remains a pressing challenge. To address this issue, we introduce a real-world dataset (ROLiD) comprising LiDAR-scanned point clouds of two random objects: water mist and smoke. In this paper, we introduce a novel adversarial perspective by proposing an attack framework that utilizes water mist and smoke to simulate environmental interference. Specifically, we propose a point cloud sequence generation method using a motion and content decomposition generative adversarial network named PCS-GAN to simulate the distribution of random objects. Furthermore, leveraging the simulated LiDAR scanning characteristics implemented with Range Image, we examine the effects of introducing random object perturbations at various positions on the target vehicle. Extensive experiments demonstrate that adversarial perturbations based on random objects effectively deceive vehicle detection and reduce the recognition rate of 3D object detection models.

Figures (7)

• Figure 1: We use random objects to camouflage the target vehicle and fool the LiDAR detector. In benign cases, the 3D object detector correctly detects the vehicle. In adversarial cases, the detector fails to identify the vehicle when random objects are superimposed on it.
• Figure 2: Our adversarial attack framework based on random object perturbations. We propose PCS-GAN to generate random object sequences ($\widetilde{P}_{T}=[\widetilde{x}^{(1)},\widetilde{x}^{(2)},\cdot \cdot \cdot,\widetilde{x}^{(K)}]$) and use a range image representation approximate renderer to simulate LiDAR scanning. We attach random objects to the target vehicle by setting different fusion modes ($m$) and fusion densities ($d_h,d_v$) to maximize the attack score of the target vehicle on LiDAR detector ($\mathcal{F}_{det}$).
• Figure 3: Random objects LiDAR Dataset (ROLiD). (a) and (b) are the data collection scheme and local point cloud visualization respectively. (c) represents the data used for the PCS-GAN network.
• Figure 4: Generative adversarial network framework PCS-GAN for point cloud sequence generation. $P$ is real point cloud, $\widetilde{x}$ is fake point cloud , $Z_C$ is content features, $Z_M$ motion features.
• Figure 5: Obtaining a range image of the point cloud via spherical projection. The first line describes the three coordinate systems involved in the conversion process, while the second line outlines the changes in data format during this transformation.