Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Spectral View of Adversarially Robust Features

Shivam Garg, Vatsal Sharan, Brian Hu Zhang, Gregory Valiant

TL;DR

Strong connections are established between adversarially robust features and a natural spectral property of the geometry of the dataset and metric of interest that can be leveraged to provide both robust features, and a lower bound on the robustness of any function that has significant variance across the dataset.

Abstract

Given the apparent difficulty of learning models that are robust to adversarial perturbations, we propose tackling the simpler problem of developing adversarially robust features. Specifically, given a dataset and metric of interest, the goal is to return a function (or multiple functions) that 1) is robust to adversarial perturbations, and 2) has significant variation across the datapoints. We establish strong connections between adversarially robust features and a natural spectral property of the geometry of the dataset and metric of interest. This connection can be leveraged to provide both robust features, and a lower bound on the robustness of any function that has significant variance across the dataset. Finally, we provide empirical evidence that the adversarially robust features given by this spectral approach can be fruitfully leveraged to learn a robust (and accurate) model.

A Spectral View of Adversarially Robust Features

TL;DR

Strong connections are established between adversarially robust features and a natural spectral property of the geometry of the dataset and metric of interest that can be leveraged to provide both robust features, and a lower bound on the robustness of any function that has significant variance across the dataset.

Abstract

Given the apparent difficulty of learning models that are robust to adversarial perturbations, we propose tackling the simpler problem of developing adversarially robust features. Specifically, given a dataset and metric of interest, the goal is to return a function (or multiple functions) that 1) is robust to adversarial perturbations, and 2) has significant variation across the datapoints. We establish strong connections between adversarially robust features and a natural spectral property of the geometry of the dataset and metric of interest. This connection can be leveraged to provide both robust features, and a lower bound on the robustness of any function that has significant variance across the dataset. Finally, we provide empirical evidence that the adversarially robust features given by this spectral approach can be fruitfully leveraged to learn a robust (and accurate) model.

Paper Structure

This paper contains 14 sections, 20 theorems, 71 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Robustness to Adversarial Perturbations
  3. Summary of Results
  4. Shortcomings and Future Work
  5. Related Work
  6. Background: Spectral Graph Theory
  7. Robust Features
  8. Extending a Feature to New Points
  9. A Lower Bound on Adversarial Robustness
  10. Synthetic Setting: Adversarial Spheres
  11. Experiments
  12. Image Classification: The MNIST Dataset
  13. Connection Between Spectral Properties and Robustness
  14. Conclusion

Key Result

Theorem 1

For any pair of datasets $X$ and $X'$, such that $dist(X, X') \le \varepsilon$, the function $F: \mathcal{X}_n \rightarrow \mathbb R^n$ obtained using the second eigenvector of the Laplacian satisfies

Figures (2)

  • Figure 1: Comparison of performance on adversarially perturbed MNIST data .
  • Figure 2: Performance on adversarial data vs our upper bound.

Theorems & Definitions (35)

  • Definition 1
  • Definition 2
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • Lemma 1
  • proof
  • Lemma 2
  • ...and 25 more