Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Eigenpatches -- Adversarial Patches from Principal Components

Jens Bayer, Stefan Becker, David Münch, Michael Arens

TL;DR

This work investigates the use of principal component analysis to create adversarial patches, coining the term eigenpatches. By analyzing 375 precalculated patches, the authors show that linear combinations of the top principal components can craft patches that substantially degrade YOLOv7 object detector performance on INRIA Person data. While PCA-reconstructed patches are generally less effective than fully trained patches, increasing the number of components amplifies the attack impact, and eight components already yield a noticeable drop in detection quality. The study provides insight into the dimensional structure of adversarial patches and suggests PCA-based patches as a potential initialization for more robust threat modeling and detector hardening, with implications for patch-based defenses on real-world detectors.

Abstract

Adversarial patches are still a simple yet powerful white box attack that can be used to fool object detectors by suppressing possible detections. The patches of these so-called evasion attacks are computational expensive to produce and require full access to the attacked detector. This paper addresses the problem of computational expensiveness by analyzing 375 generated patches, calculating the principal components of these and show, that linear combinations of the resulting "eigenpatches" can be used to fool object detections successfully.

Eigenpatches -- Adversarial Patches from Principal Components

TL;DR

This work investigates the use of principal component analysis to create adversarial patches, coining the term eigenpatches. By analyzing 375 precalculated patches, the authors show that linear combinations of the top principal components can craft patches that substantially degrade YOLOv7 object detector performance on INRIA Person data. While PCA-reconstructed patches are generally less effective than fully trained patches, increasing the number of components amplifies the attack impact, and eight components already yield a noticeable drop in detection quality. The study provides insight into the dimensional structure of adversarial patches and suggests PCA-based patches as a potential initialization for more robust threat modeling and detector hardening, with implications for patch-based defenses on real-world detectors.

Abstract

Adversarial patches are still a simple yet powerful white box attack that can be used to fool object detectors by suppressing possible detections. The patches of these so-called evasion attacks are computational expensive to produce and require full access to the attacked detector. This paper addresses the problem of computational expensiveness by analyzing 375 generated patches, calculating the principal components of these and show, that linear combinations of the resulting "eigenpatches" can be used to fool object detections successfully.
Paper Structure (12 sections, 2 equations, 5 figures, 2 tables)

This paper contains 12 sections, 2 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Eigenpatches
  4. Experimental Setup
  5. Object Detector
  6. Dataset
  7. Patch Generation
  8. Evaluation
  9. Metrics
  10. Experimental Setup
  11. Results
  12. Conclusion

Figures (5)

  • Figure 1: Left: the first sixteen principal components extracted from the trained patches. Right: the unweighted mean image of these sixteen principal components.
  • Figure 2: Patched input image as it would be used in the evaluation. The image is resized and padded to match the required input size of $640\times640$ pixels. After this, the tested patch is embedded in the center of each bounding box and resized according to a scaling factor.
  • Figure 3: Drop of the mAP@.5 and mAP@.5:.95 with different numbers of principal components to the recovered patches.
  • Figure 4: mAP@.5 and mAP@.5:.95 curve with different numbers of input elements for the PCA to recreate the patches with $\min(n,64)$ principal components.
  • Figure 5: Left: The input set for the PCA. Middle: The computed principal components. Right: Example recreations of the patches by the two principal components.