Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Fingerprints for Adversarial Attack Detection

Haim Fisher, Moni Shahar, Yehezkel S. Resheff

TL;DR

This paper argues that in a white-box setting, where the attacker knows the configuration and weights of the network and the detector, they can overcome the detector by running many examples on a local copy, and sending only those that were not detected to the actual model.

Abstract

Deep learning models for image classification have become standard tools in recent years. A well known vulnerability of these models is their susceptibility to adversarial examples. These are generated by slightly altering an image of a certain class in a way that is imperceptible to humans but causes the model to classify it wrongly as another class. Many algorithms have been proposed to address this problem, falling generally into one of two categories: (i) building robust classifiers (ii) directly detecting attacked images. Despite the good performance of these detectors, we argue that in a white-box setting, where the attacker knows the configuration and weights of the network and the detector, they can overcome the detector by running many examples on a local copy, and sending only those that were not detected to the actual model. This problem is common in security applications where even a very good model is not sufficient to ensure safety. In this paper we propose to overcome this inherent limitation of any static defence with randomization. To do so, one must generate a very large family of detectors with consistent performance, and select one or more of them randomly for each input. For the individual detectors, we suggest the method of neural fingerprints. In the training phase, for each class we repeatedly sample a tiny random subset of neurons from certain layers of the network, and if their average is sufficiently different between clean and attacked images of the focal class they are considered a fingerprint and added to the detector bank. During test time, we sample fingerprints from the bank associated with the label predicted by the model, and detect attacks using a likelihood ratio test. We evaluate our detectors on ImageNet with different attack methods and model architectures, and show near-perfect detection with low rates of false detection.

Neural Fingerprints for Adversarial Attack Detection

TL;DR

This paper argues that in a white-box setting, where the attacker knows the configuration and weights of the network and the detector, they can overcome the detector by running many examples on a local copy, and sending only those that were not detected to the actual model.

Abstract

Deep learning models for image classification have become standard tools in recent years. A well known vulnerability of these models is their susceptibility to adversarial examples. These are generated by slightly altering an image of a certain class in a way that is imperceptible to humans but causes the model to classify it wrongly as another class. Many algorithms have been proposed to address this problem, falling generally into one of two categories: (i) building robust classifiers (ii) directly detecting attacked images. Despite the good performance of these detectors, we argue that in a white-box setting, where the attacker knows the configuration and weights of the network and the detector, they can overcome the detector by running many examples on a local copy, and sending only those that were not detected to the actual model. This problem is common in security applications where even a very good model is not sufficient to ensure safety. In this paper we propose to overcome this inherent limitation of any static defence with randomization. To do so, one must generate a very large family of detectors with consistent performance, and select one or more of them randomly for each input. For the individual detectors, we suggest the method of neural fingerprints. In the training phase, for each class we repeatedly sample a tiny random subset of neurons from certain layers of the network, and if their average is sufficiently different between clean and attacked images of the focal class they are considered a fingerprint and added to the detector bank. During test time, we sample fingerprints from the bank associated with the label predicted by the model, and detect attacks using a likelihood ratio test. We evaluate our detectors on ImageNet with different attack methods and model architectures, and show near-perfect detection with low rates of false detection.

Paper Structure

This paper contains 10 sections, 12 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background: adversarial attacks and defence strategies
  3. Adversarial Attacks
  4. Defences
  5. The method of Neural Fingerprints
  6. Neural Fingerprints
  7. Related Work
  8. Results
  9. Conclusion
  10. Appendix A: attacked images

Figures (3)

  • Figure 1: Fingerprint distributions: (a) randomly selected fingerprints (b) fingerprints filtered based on effect size. Orange - clean image, blue - attacked.
  • Figure 2: Detection AUC as a function of Number of Fingerprints for the three detection methods: Vote, Anomaly, and Likelihood for both models.
  • Figure 3: Example of original images and their adversarial attacks. The first row shows original images of classes goldfish, scorpion, snake, and toucan. The subsequent rows demonstrate IFGSM attacks on Iception V3, of each original image to three other categories: Banana, Printer, Tiger-Shark.