A Real-Time Defense Against Object Vanishing Adversarial Patch Attacks for Object Detection in Autonomous Vehicles

TL;DR

The paper tackles the threat of object vanishing adversarial patches in autonomous-vehicle perception by introducing ADAV, a real-time defense that leverages temporal consistency across video frames. It employs a two-stage pipeline: first detecting patches by comparing the current frame's detector outputs to a clean reference frame from $0.5$ seconds prior, then localizing and masking adversarial pixels using gradient-based attribution if a patch is detected. Evaluations on the BDD100K dataset show that ADAV improves adversarial robustness while preserving clean detection performance and maintaining real-time inference speed. This approach enhances safety and trust in AV perception by effectively mitigating patch-based attacks without substantial latency penalties.

Abstract

Autonomous vehicles (AVs) increasingly use DNN-based object detection models in vision-based perception. Correct detection and classification of obstacles is critical to ensure safe, trustworthy driving decisions. Adversarial patches aim to fool a DNN with intentionally generated patterns concentrated in a localized region of an image. In particular, object vanishing patch attacks can cause object detection models to fail to detect most or all objects in a scene, posing a significant practical threat to AVs. This work proposes ADAV (Adversarial Defense for Autonomous Vehicles), a novel defense methodology against object vanishing patch attacks specifically designed for autonomous vehicles. Unlike existing defense methods which have high latency or are designed for static images, ADAV runs in real-time and leverages contextual information from prior frames in an AV's video feed. ADAV checks if the object detector's output for the target frame is temporally consistent with the output from a previous reference frame to detect the presence of a patch. If the presence of a patch is detected, ADAV uses gradient-based attribution to localize adversarial pixels that break temporal consistency. This two stage procedure allows ADAV to efficiently process clean inputs, and both stages are optimized to be low latency. ADAV is evaluated using real-world driving data from the Berkeley Deep Drive BDD100K dataset, and demonstrates high adversarial and clean performance.

Figures (4)

• Figure 1: The Object Vanishing Adversarial Patch
• Figure 2: ADAV Methodology
• Figure 3: Adversarial vs Clean Performance
• Figure 4: Adversarial Patch