Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Finer Calibration Analysis for Adversarial Robustness

Pranjal Awasthi, Anqi Mao, Mehryar Mohri, Yutao Zhong

TL;DR

The paper tackles ${\mathscr H}$-calibration for adversarially robust binary classification by clarifying non-uniform versus uniform calibration and correcting prior claims about ${\mathscr H}$-consistency. It shows that common convex surrogate losses fail ${\mathscr H}$-calibration with respect to the adversarial loss $\ell_{\gamma}$, while carefully constructed non-convex margin-based and $\rho$-margin losses can be calibrated and, under realizability, are ${\mathscr H}$-consistent. The results hold for broad hypothesis sets, including generalized linear models and ReLU networks, extending and strengthening previous work (Bao 2020; PNAMY 2021) and removing restrictive unboundedness assumptions. Together, these findings guide the design of theoretically sound surrogate losses for adversarial robustness across a wide range of models and settings.

Abstract

We present a more general analysis of $H$-calibration for adversarially robust classification. By adopting a finer definition of calibration, we can cover settings beyond the restricted hypothesis sets studied in previous work. In particular, our results hold for most common hypothesis sets used in machine learning. We both fix some previous calibration results (Bao et al., 2020) and generalize others (Awasthi et al., 2021). Moreover, our calibration results, combined with the previous study of consistency by Awasthi et al. (2021), also lead to more general $H$-consistency results covering common hypothesis sets.

A Finer Calibration Analysis for Adversarial Robustness

TL;DR

The paper tackles -calibration for adversarially robust binary classification by clarifying non-uniform versus uniform calibration and correcting prior claims about -consistency. It shows that common convex surrogate losses fail -calibration with respect to the adversarial loss , while carefully constructed non-convex margin-based and -margin losses can be calibrated and, under realizability, are -consistent. The results hold for broad hypothesis sets, including generalized linear models and ReLU networks, extending and strengthening previous work (Bao 2020; PNAMY 2021) and removing restrictive unboundedness assumptions. Together, these findings guide the design of theoretically sound surrogate losses for adversarial robustness across a wide range of models and settings.

Abstract

We present a more general analysis of -calibration for adversarially robust classification. By adopting a finer definition of calibration, we can cover settings beyond the restricted hypothesis sets studied in previous work. In particular, our results hold for most common hypothesis sets used in machine learning. We both fix some previous calibration results (Bao et al., 2020) and generalize others (Awasthi et al., 2021). Moreover, our calibration results, combined with the previous study of consistency by Awasthi et al. (2021), also lead to more general -consistency results covering common hypothesis sets.

Paper Structure

This paper contains 17 sections, 29 theorems, 144 equations.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Adversarially Robust Classification
  4. H-Calibration Analysis
  5. Negative results
  6. Convex losses
  7. Supremum-based convex losses
  8. Positive results
  9. Margin-based losses
  10. Supremum-based margin losses
  11. ${\mathscr H}$-consistency
  12. Conclusion
  13. Deferred Proofs
  14. Proof of Theorem \ref{['Thm:calibration_margin_convex']}, Theorem \ref{['Thm:linear_sup_convex']} and Theorem \ref{['Thm:calibration_sup_convex']}
  15. Property of BarC
  16. ...and 2 more sections

Key Result

Proposition 5

Given a hypothesis set ${\mathscr H}$, loss $\ell_1$ is ${\mathscr H}$-calibrated with respect to $\ell_2$ if and only if its calibration function $\delta_{\max}$ satisfies $\delta_{\max}(\epsilon,{\mathbf x},\eta)>0$ for all ${\mathbf x}\in {\mathscr X}$, $\eta\in [0,1]$ and $\epsilon>0$.

Theorems & Definitions (34)

  • Definition 1: ${\mathscr H}$-Consistency
  • Definition 2: ${\mathscr H}$-Calibration
  • Definition 3: Uniform ${\mathscr H}$-Calibration
  • Definition 4: Calibration function
  • Proposition 5: Lemma 2.9 in steinwart2007compare
  • Definition 6: Regularity for Adversarial Calibration
  • Theorem 7
  • Theorem 8
  • Corollary 9
  • Theorem 10
  • ...and 24 more