Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Existence and Minimax Theorems for Adversarial Surrogate Risks in Binary Classification

Natalie S. Frank, Jonathan Niles-Weed

TL;DR

The results explain some empirical observations on adversarial robustness from prior work and suggest new directions in algorithm development.

Abstract

Adversarial training is one of the most popular methods for training methods robust to adversarial attacks, however, it is not well-understood from a theoretical perspective. We prove and existence, regularity, and minimax theorems for adversarial surrogate risks. Our results explain some empirical observations on adversarial robustness from prior work and suggest new directions in algorithm development. Furthermore, our results extend previously known existence and minimax theorems for the adversarial classification risk to surrogate risks.

Existence and Minimax Theorems for Adversarial Surrogate Risks in Binary Classification

TL;DR

The results explain some empirical observations on adversarial robustness from prior work and suggest new directions in algorithm development.

Abstract

Adversarial training is one of the most popular methods for training methods robust to adversarial attacks, however, it is not well-understood from a theoretical perspective. We prove and existence, regularity, and minimax theorems for adversarial surrogate risks. Our results explain some empirical observations on adversarial robustness from prior work and suggest new directions in algorithm development. Furthermore, our results extend previously known existence and minimax theorems for the adversarial classification risk to surrogate risks.
Paper Structure (30 sections, 57 theorems, 149 equations)

This paper contains 30 sections, 57 theorems, 149 equations.

Table of Contents

  1. Introduction
  2. Related Works
  3. Background and Notation
  4. Adversarial Classification
  5. Surrogate Risks
  6. Measurability
  7. The $W_\infty$ metric
  8. Main Results and Outline of the Paper
  9. Summary of Main Results
  10. Outline of Main Argument
  11. Paper Outline
  12. A Duality Result for $\Theta$ and $\bar{R}_\phi$
  13. Strong Duality
  14. Complimentary Slackness
  15. Existence of minimizers of $\Theta$ over $S_\psi$
  16. ...and 15 more sections

Key Result

Theorem 1

If $f$ is universally measurable, then $S_\epsilon(f)$ is also universally measurable.

Theorems & Definitions (59)

  • Theorem 1
  • Definition 2
  • Lemma 2
  • Lemma 2
  • Theorem 3: PydiJog2021
  • Theorem 4: Strong Duality
  • Theorem 5: Complimentary Slackness
  • Theorem 6
  • Theorem 7
  • Theorem 8: Fenchel-Rockafellar Duality Theorem
  • ...and 49 more