Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

I Don't Know You, But I Can Catch You: Real-Time Defense against Diverse Adversarial Patches for Object Detectors

Zijin Lin, Yue Zhao, Kai Chen, Jinwen He

TL;DR

The proposed NutNet is an innovative model for detecting adversarial patches, with high generalization, robustness and efficiency, and exhibits an average defense performance that is over 2.4 times and 4.7 times higher than existing approaches for HA and AA, respectively.

Abstract

Deep neural networks (DNNs) have revolutionized the field of computer vision like object detection with their unparalleled performance. However, existing research has shown that DNNs are vulnerable to adversarial attacks. In the physical world, an adversary could exploit adversarial patches to implement a Hiding Attack (HA) which patches the target object to make it disappear from the detector, and an Appearing Attack (AA) which fools the detector into misclassifying the patch as a specific object. Recently, many defense methods for detectors have been proposed to mitigate the potential threats of adversarial patches. However, such methods still have limitations in generalization, robustness and efficiency. Most defenses are only effective against the HA, leaving the detector vulnerable to the AA. In this paper, we propose \textit{NutNet}, an innovative model for detecting adversarial patches, with high generalization, robustness and efficiency. With experiments for six detectors including YOLOv2-v4, SSD, Faster RCNN and DETR on both digital and physical domains, the results show that our proposed method can effectively defend against both the HA and AA, with only 0.4\% sacrifice of the clean performance. We compare NutNet with four baseline defense methods for detectors, and our method exhibits an average defense performance that is over 2.4 times and 4.7 times higher than existing approaches for HA and AA, respectively. In addition, NutNet only increases the inference time by 8\%, which can meet the real-time requirements of the detection systems. Demos of NutNet are available at: \url{https://sites.google.com/view/nutnet}.

I Don't Know You, But I Can Catch You: Real-Time Defense against Diverse Adversarial Patches for Object Detectors

TL;DR

The proposed NutNet is an innovative model for detecting adversarial patches, with high generalization, robustness and efficiency, and exhibits an average defense performance that is over 2.4 times and 4.7 times higher than existing approaches for HA and AA, respectively.

Abstract

Deep neural networks (DNNs) have revolutionized the field of computer vision like object detection with their unparalleled performance. However, existing research has shown that DNNs are vulnerable to adversarial attacks. In the physical world, an adversary could exploit adversarial patches to implement a Hiding Attack (HA) which patches the target object to make it disappear from the detector, and an Appearing Attack (AA) which fools the detector into misclassifying the patch as a specific object. Recently, many defense methods for detectors have been proposed to mitigate the potential threats of adversarial patches. However, such methods still have limitations in generalization, robustness and efficiency. Most defenses are only effective against the HA, leaving the detector vulnerable to the AA. In this paper, we propose \textit{NutNet}, an innovative model for detecting adversarial patches, with high generalization, robustness and efficiency. With experiments for six detectors including YOLOv2-v4, SSD, Faster RCNN and DETR on both digital and physical domains, the results show that our proposed method can effectively defend against both the HA and AA, with only 0.4\% sacrifice of the clean performance. We compare NutNet with four baseline defense methods for detectors, and our method exhibits an average defense performance that is over 2.4 times and 4.7 times higher than existing approaches for HA and AA, respectively. In addition, NutNet only increases the inference time by 8\%, which can meet the real-time requirements of the detection systems. Demos of NutNet are available at: \url{https://sites.google.com/view/nutnet}.
Paper Structure (29 sections, 3 equations, 8 figures, 14 tables)

This paper contains 29 sections, 3 equations, 8 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Object Detection
  4. Adversarial Attacks
  5. Adversarial Defense
  6. Overview
  7. Threat Model
  8. Motivation and Defense Intuition
  9. Overview of NutNet
  10. Approach
  11. NutNet
  12. The Localization and Mitigation of Patches
  13. Evaluation
  14. Experimental Setup
  15. Effectiveness
  16. ...and 14 more sections

Figures (8)

  • Figure 1: The framework of our defense.
  • Figure 2: Examples of detection results on images with adversarial patches with NutNet. The top images show predictions on the original patched images, while the bottom images show predictions on the patched images with NutNet.
  • Figure 3: The inner product between the gradient of attacking the object detector and the gradient of bypassing NutNet.
  • Figure 4: Examples of the defensive performance of SAC (column 2), Jedi (column 3) and NutNet (column 4). The first column shows the original patched images. Note that SAC uses black masks while Jedi and NutNet use gray ones.
  • Figure 5: Efficiency (FPS) of different defense methods for different detectors running on KITTI dataset.
  • ...and 3 more figures