Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Fair Robustness via Domain Mixup

Meiyu Zhong, Ravi Tandon

TL;DR

This paper presents a theoretical analysis of the use of mixup for the problem of learning fair robust classifiers and shows that mixup combined with adversarial training can provably reduce the class-wise robustness disparity.

Abstract

Adversarial training is one of the predominant techniques for training classifiers that are robust to adversarial attacks. Recent work, however has found that adversarial training, which makes the overall classifier robust, it does not necessarily provide equal amount of robustness for all classes. In this paper, we propose the use of mixup for the problem of learning fair robust classifiers, which can provide similar robustness across all classes. Specifically, the idea is to mix inputs from the same classes and perform adversarial training on mixed up inputs. We present a theoretical analysis of this idea for the case of linear classifiers and show that mixup combined with adversarial training can provably reduce the class-wise robustness disparity. This method not only contributes to reducing the disparity in class-wise adversarial risk, but also the class-wise natural risk. Complementing our theoretical analysis, we also provide experimental results on both synthetic data and the real world dataset (CIFAR-10), which shows improvement in class wise disparities for both natural and adversarial risks.

Learning Fair Robustness via Domain Mixup

TL;DR

This paper presents a theoretical analysis of the use of mixup for the problem of learning fair robust classifiers and shows that mixup combined with adversarial training can provably reduce the class-wise robustness disparity.

Abstract

Adversarial training is one of the predominant techniques for training classifiers that are robust to adversarial attacks. Recent work, however has found that adversarial training, which makes the overall classifier robust, it does not necessarily provide equal amount of robustness for all classes. In this paper, we propose the use of mixup for the problem of learning fair robust classifiers, which can provide similar robustness across all classes. Specifically, the idea is to mix inputs from the same classes and perform adversarial training on mixed up inputs. We present a theoretical analysis of this idea for the case of linear classifiers and show that mixup combined with adversarial training can provably reduce the class-wise robustness disparity. This method not only contributes to reducing the disparity in class-wise adversarial risk, but also the class-wise natural risk. Complementing our theoretical analysis, we also provide experimental results on both synthetic data and the real world dataset (CIFAR-10), which shows improvement in class wise disparities for both natural and adversarial risks.

Paper Structure

This paper contains 6 sections, 4 theorems, 48 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries and Problem Statement
  3. Main Results and Discussion
  4. Conclusion
  5. Proof of Theorem \ref{['the:mixupnat']}
  6. Proof of Theorem \ref{['the: advmixp']}

Key Result

Proposition 1

xu2021robustma2022tradeoff (Class Wise Natural Risk) For the optimized linear classifier $f^*_{\text{nat}}$ that minimizes the natural risk, when $\sigma_{-} = \sigma_{+} = \sigma$, the class wise risks are given as: where K is a positive constant.

Figures (2)

  • Figure 1: (a) Comparison of the natural risk and the natural domain mixup risk with the Gaussian data using a linear model with respect to the distance of two classes ($|\mu_{+}+\mu_{-}|$). We can observe that domain mixup mechanism consistently decreases the discrepancy of the class wise risk as we increase the distance (difference in mean) across the two classes. (b) Comparison of the adversarial risk and the adversarial domain mixup risk with the Gaussian data using a linear model as we increase the value of $\epsilon$. We can observe that domain mixup mechanism consistently reduces the gap of the class wise adversarial risk.
  • Figure 2: (a) Comparison of the adversarial risk and the adversarial domain mixup risk with the Gaussian data using a linear model with respect to the distance of two classes ($|\mu_{+}+\mu_{-}|$). We can observe that domain mixup mechanism consistently decreases the discrepancy of the class wise risk as we increase the distance of two classes. (b) Comparison of the adversarial risk and the adversarial domain mixup risk with the Gaussian data using a linear model as we increase the dimension d. We note that Mixup mechanism consistently reduces the gap of the class wise adversarial risk when we increase the data dimension.

Theorems & Definitions (11)

  • Definition 1
  • Proposition 1
  • Proposition 2
  • Theorem 3
  • Remark 1
  • Remark 2
  • Theorem 4
  • Remark 3
  • Remark 4
  • proof
  • ...and 1 more