DPatch: An Adversarial Patch Attack on Object Detectors

TL;DR

This work addresses the vulnerability of modern object detectors to physical-world adversarial patches by introducing DPatch, a small 40x40 patch trained to disrupt both region proposals and classification in Faster R-CNN and YOLO under black-box conditions. It formalizes untargeted and targeted objectives with dedicated training goals and random shifts to enforce location invariance, and demonstrates that the patch can drastically reduce mean average precision (mAP) on multiple detectors and datasets. The results show strong cross-detector and cross-dataset transferability, patch-size effects, and ROI concentration around the patch, underscoring a fundamental security risk in contemporary detection systems. The findings motivate the development of robust defenses for detectors used in safety-critical applications such as surveillance and autonomous driving.

Abstract

Object detectors have emerged as an indispensable module in modern computer vision systems. In this work, we propose DPatch -- a black-box adversarial-patch-based attack towards mainstream object detectors (i.e. Faster R-CNN and YOLO). Unlike the original adversarial patch that only manipulates image-level classifier, our DPatch simultaneously attacks the bounding box regression and object classification so as to disable their predictions. Compared to prior works, DPatch has several appealing properties: (1) DPatch can perform both untargeted and targeted effective attacks, degrading the mAP of Faster R-CNN and YOLO from 75.10% and 65.7% down to below 1%, respectively. (2) DPatch is small in size and its attacking effect is location-independent, making it very practical to implement real-world attacks. (3) DPatch demonstrates great transferability among different detectors as well as training datasets. For example, DPatch that is trained on Faster R-CNN can effectively attack YOLO, and vice versa. Extensive evaluations imply that DPatch can perform effective attacks under black-box setup, i.e., even without the knowledge of the attacked network's architectures and parameters. Successful realization of DPatch also illustrates the intrinsic vulnerability of the modern detector architectures to such patch-based adversarial attacks.

Figures (11)

• Figure 1: The original adversarial patch fails to attack object detectors. The first row is the original image. Faster R-CNN can detect multiple objects in the scene with a high accuracy. The second row is the image embedded with the Google's adversarial patch whose targeted class is $toaster$. Faster R-CNN is not influenced by the patch and can still be able to correctly recognize all the objects.
• Figure 2: DPatch training system: we add a randomly-initialized DPatch to the image, utilize the detector network to do classification and bounding box regression based on the ground truth [DPatch_x, DPatch_y, DPatch_w, DPatch_h, target_label]. During back-propagation, we update the pixels of DPatch.
• Figure 3: two DPatch es.
• Figure 4: YOLO cannot detect $bike$ after adding DPatch Either the predicted bounding box or the classification result is incorrect. The predicted bounding box is just a dot, while the classification result is $sofa$ and $person$.
• Figure 5: The DPatch is placed on the left top corner of the images. Faster R-CNN networks can on longer recognize other objects. Here, we apply a DPatch whose targeted class is $tv$, it can fully fool the Faster R-CNN networks though the DPatch size is comparatively small compared to other objects in the scene.