Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Anatomy of Adversarial Attacks: Concept-based XAI Dissection

Georgii Mikriukov, Gesina Schwalbe, Franz Motzkus, Korinna Bade

TL;DR

An in-depth analysis of the influence of AAs on the concepts learned by convolutional neural networks (CNNs) using eXplainable artificial intelligence (XAI) techniques provides valuable insights into the nature of AAs and their impact on learned representations, paving the way for the development of more robust and interpretable deep learning models, as well as effective defenses against adversarial threats.

Abstract

Adversarial attacks (AAs) pose a significant threat to the reliability and robustness of deep neural networks. While the impact of these attacks on model predictions has been extensively studied, their effect on the learned representations and concepts within these models remains largely unexplored. In this work, we perform an in-depth analysis of the influence of AAs on the concepts learned by convolutional neural networks (CNNs) using eXplainable artificial intelligence (XAI) techniques. Through an extensive set of experiments across various network architectures and targeted AA techniques, we unveil several key findings. First, AAs induce substantial alterations in the concept composition within the feature space, introducing new concepts or modifying existing ones. Second, the adversarial perturbation itself can be linearly decomposed into a set of latent vector components, with a subset of these being responsible for the attack's success. Notably, we discover that these components are target-specific, i.e., are similar for a given target class throughout different AA techniques and starting classes. Our findings provide valuable insights into the nature of AAs and their impact on learned representations, paving the way for the development of more robust and interpretable deep learning models, as well as effective defenses against adversarial threats.

The Anatomy of Adversarial Attacks: Concept-based XAI Dissection

TL;DR

An in-depth analysis of the influence of AAs on the concepts learned by convolutional neural networks (CNNs) using eXplainable artificial intelligence (XAI) techniques provides valuable insights into the nature of AAs and their impact on learned representations, paving the way for the development of more robust and interpretable deep learning models, as well as effective defenses against adversarial threats.

Abstract

Adversarial attacks (AAs) pose a significant threat to the reliability and robustness of deep neural networks. While the impact of these attacks on model predictions has been extensively studied, their effect on the learned representations and concepts within these models remains largely unexplored. In this work, we perform an in-depth analysis of the influence of AAs on the concepts learned by convolutional neural networks (CNNs) using eXplainable artificial intelligence (XAI) techniques. Through an extensive set of experiments across various network architectures and targeted AA techniques, we unveil several key findings. First, AAs induce substantial alterations in the concept composition within the feature space, introducing new concepts or modifying existing ones. Second, the adversarial perturbation itself can be linearly decomposed into a set of latent vector components, with a subset of these being responsible for the attack's success. Notably, we discover that these components are target-specific, i.e., are similar for a given target class throughout different AA techniques and starting classes. Our findings provide valuable insights into the nature of AAs and their impact on learned representations, paving the way for the development of more robust and interpretable deep learning models, as well as effective defenses against adversarial threats.
Paper Structure (45 sections, 10 equations, 9 figures, 2 tables)

This paper contains 45 sections, 10 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Adversarial attacks
  4. Concept-based XAI Methods
  5. Adversarial Attacks and XAI
  6. Background
  7. Adversarial Attacks
  8. BIM
  9. PGD
  10. C&W
  11. Adversarial Patch
  12. Concept Discovery with Matrix Factorization
  13. PCA
  14. NMF
  15. Concept Comparison
  16. ...and 30 more sections

Figures (9)

  • Figure 1: Examples of BIM, PGD, C&W, and Patch Attack adversarial samples: "fire truck" attacked with target "banana".
  • Figure 2: Mean and standard deviation values of cosine similarities for original and attacked activation maps of test samples for several attacks.
  • Figure 3: Mean and standard deviation values of cosine similarities for original and attacked activation maps of test samples for several attacks.
  • Figure 4: Concept mining results for BIM and C&W ($\texttt{fire truck} \rightarrow \texttt{taxi}$) attacks in layer $layer4.0$ of ResNet18. Top: pairs of top-2 most relevant prototypes of discovered concepts cX with rank X and concept weights (importances); Bottom: discovered concept similarities (Sec. \ref{['sec:background-comparison']}) for original vs. BIM (left) and original vs. C&W (right)
  • Figure 5: Mean numbers of "concept changes" with 99% confidence intervals for threshold values 75, 50, and 25 in all tested models for tested adversarial attacks.
  • ...and 4 more figures