Dynamic Adversarial Attacks on Autonomous Driving Systems

TL;DR

This paper introduces a novel adversarial attack approach where the patch is not co-located with its target, enabling more versatile and stealthy attacks and designs a Screen Image Transformation Network (SIT-Net), which simulates environmental effects on the displayed images, narrowing the gap between simulated and real-world scenarios.

Abstract

This paper introduces an attacking mechanism to challenge the resilience of autonomous driving systems. Specifically, we manipulate the decision-making processes of an autonomous vehicle by dynamically displaying adversarial patches on a screen mounted on another moving vehicle. These patches are optimized to deceive the object detection models into misclassifying targeted objects, e.g., traffic signs. Such manipulation has significant implications for critical multi-vehicle interactions such as intersection crossing and lane changing, which are vital for safe and efficient autonomous driving systems. Particularly, we make four major contributions. First, we introduce a novel adversarial attack approach where the patch is not co-located with its target, enabling more versatile and stealthy attacks. Moreover, our method utilizes dynamic patches displayed on a screen, allowing for adaptive changes and movement, enhancing the flexibility and performance of the attack. To do so, we design a Screen Image Transformation Network (SIT-Net), which simulates environmental effects on the displayed images, narrowing the gap between simulated and real-world scenarios. Further, we integrate a positional loss term into the adversarial training process to increase the success rate of the dynamic attack. Finally, we shift the focus from merely attacking perceptual systems to influencing the decision-making algorithms of self-driving systems. Our experiments demonstrate the first successful implementation of such dynamic adversarial attacks in real-world autonomous driving scenarios, paving the way for advancements in the field of robust and secure autonomous driving.

Figures (17)

• Figure 1: Sequential frames of a multi-robot interaction in an intersection: The top row of (a) demonstrates a dynamic adversarial patch attack, where the patch is displayed on the screen of the patch car. The patch is designed to mislead the camera car's perception system, i.e., the camera car thinks the Pedestrian sign is a Stop sign. The top row of (b) illustrates the system's correct behavior during a benign trial without adversarial interference. The bottom rows of (a) and (b) are the top views of the settings.
• Figure 2: The position of objects in an image influences the performance of adversarial patches. The influence, as visualized in heatmaps, shifts according to changes in objects' positions. The color of a heatmap reflects the influence level (red to purple denoting most to least influence).
• Figure 3: Analysis of data clustering based on positions: (a) Lidar map showing the coordinates of the cars and the target in a single frame; (b) Cluster plot of the patch car and the target for all frames in a dataset.
• Figure 4: SIT-Net predictions and the loss during its training.
• Figure 5: The patch optimization pipeline.