Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring Layerwise Adversarial Robustness Through the Lens of t-SNE

Inês Valentim, Nuno Antunes, Nuno Lourenço

TL;DR

Adversarial examples threaten image classifiers, and the paper introduces a t-SNE-based visualization paired with a clean-perturbed embedding overlap metric to diagnose layerwise robustness on CIFAR-10. The method reveals that discrepancies between clean and perturbed representations emerge early in feature extraction, with metric values lying in $[0,1]$ indicating the degree of overlap. Across WRN-28-10 and DENSER under $L_2$ and $L_$ attacks, robustness declines in a layer-dependent manner that aligns with separations observed in 2D embeddings. This diagnostic approach offers a practical tool to guide defense design and detector-layer selection, though generalizability is limited by using a small set of pre-trained models and datasets.

Abstract

Adversarial examples, designed to trick Artificial Neural Networks (ANNs) into producing wrong outputs, highlight vulnerabilities in these models. Exploring these weaknesses is crucial for developing defenses, and so, we propose a method to assess the adversarial robustness of image-classifying ANNs. The t-distributed Stochastic Neighbor Embedding (t-SNE) technique is used for visual inspection, and a metric, which compares the clean and perturbed embeddings, helps pinpoint weak spots in the layers. Analyzing two ANNs on CIFAR-10, one designed by humans and another via NeuroEvolution, we found that differences between clean and perturbed representations emerge early on, in the feature extraction layers, affecting subsequent classification. The findings with our metric are supported by the visual analysis of the t-SNE maps.

Exploring Layerwise Adversarial Robustness Through the Lens of t-SNE

TL;DR

Adversarial examples threaten image classifiers, and the paper introduces a t-SNE-based visualization paired with a clean-perturbed embedding overlap metric to diagnose layerwise robustness on CIFAR-10. The method reveals that discrepancies between clean and perturbed representations emerge early in feature extraction, with metric values lying in indicating the degree of overlap. Across WRN-28-10 and DENSER under and attacks, robustness declines in a layer-dependent manner that aligns with separations observed in 2D embeddings. This diagnostic approach offers a practical tool to guide defense design and detector-layer selection, though generalizability is limited by using a small set of pre-trained models and datasets.

Abstract

Adversarial examples, designed to trick Artificial Neural Networks (ANNs) into producing wrong outputs, highlight vulnerabilities in these models. Exploring these weaknesses is crucial for developing defenses, and so, we propose a method to assess the adversarial robustness of image-classifying ANNs. The t-distributed Stochastic Neighbor Embedding (t-SNE) technique is used for visual inspection, and a metric, which compares the clean and perturbed embeddings, helps pinpoint weak spots in the layers. Analyzing two ANNs on CIFAR-10, one designed by humans and another via NeuroEvolution, we found that differences between clean and perturbed representations emerge early on, in the feature extraction layers, affecting subsequent classification. The findings with our metric are supported by the visual analysis of the t-SNE maps.
Paper Structure (11 sections, 1 equation, 4 figures)

This paper contains 11 sections, 1 equation, 4 figures.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Methodology
  4. Robustness Metric
  5. Experimental Setup
  6. Dataset and Models
  7. Threat Models, Attacks, and t-SNE
  8. Inspecting the Networks
  9. Robustness Metric across Layers
  10. Visual Inspection
  11. Conclusion and Future Work

Figures (4)

  • Figure 1: Proposed methodology to analyze the adversarial robustness of different layers of a CNN.
  • Figure 2: Layerwise robustness of the models under APGD-CE with $L_2$ (blue) and $L_\infty$ (orange) perturbations.
  • Figure 3: Non-overlapping clean (green) and perturbed (red) points on the t-SNE map of different intermediate layers of the DENSER model, considering an APGD-CE attack in $L_2$.
  • Figure 4: t-SNE map for the last layer of DENSER and an APGD-CE attack in $L_2$. Colors represent true labels.