Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Multimarginal Optimal Transport Formulation of Adversarial Multiclass Classification

Nicolas Garcia Trillos, Matt Jacobs, Jakwang Kim

TL;DR

A family of adversarial multiclass classification problems are studied to recover the optimal robust classification rule and the optimal adversarial strategy for the original adversarial problem by solving either the barycenter problem or the MOT problem.

Abstract

We study a family of adversarial multiclass classification problems and provide equivalent reformulations in terms of: 1) a family of generalized barycenter problems introduced in the paper and 2) a family of multimarginal optimal transport problems where the number of marginals is equal to the number of classes in the original classification problem. These new theoretical results reveal a rich geometric structure of adversarial learning problems in multiclass classification and extend recent results restricted to the binary classification setting. A direct computational implication of our results is that by solving either the barycenter problem and its dual, or the MOT problem and its dual, we can recover the optimal robust classification rule and the optimal adversarial strategy for the original adversarial problem. Examples with synthetic and real data illustrate our results.

The Multimarginal Optimal Transport Formulation of Adversarial Multiclass Classification

TL;DR

A family of adversarial multiclass classification problems are studied to recover the optimal robust classification rule and the optimal adversarial strategy for the original adversarial problem by solving either the barycenter problem or the MOT problem.

Abstract

We study a family of adversarial multiclass classification problems and provide equivalent reformulations in terms of: 1) a family of generalized barycenter problems introduced in the paper and 2) a family of multimarginal optimal transport problems where the number of marginals is equal to the number of classes in the original classification problem. These new theoretical results reveal a rich geometric structure of adversarial learning problems in multiclass classification and extend recent results restricted to the binary classification setting. A direct computational implication of our results is that by solving either the barycenter problem and its dual, or the MOT problem and its dual, we can recover the optimal robust classification rule and the optimal adversarial strategy for the original adversarial problem. Examples with synthetic and real data illustrate our results.
Paper Structure (20 sections, 21 theorems, 232 equations, 6 figures)

This paper contains 20 sections, 21 theorems, 232 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Outline of paper
  3. Preliminaries
  4. The MOT problem
  5. General MOT problems
  6. From adversarial robustness to MOT
  7. The generalized barycenter problem
  8. A first MOT reformulation of \ref{['eq:lambda_decomposed_formulation']} and geometric consequences
  9. A second MOT reformulation of \ref{['eq:lambda_decomposed_formulation']}
  10. Localization
  11. Dual Problems
  12. Proof of Theorem \ref{['thm:Main']}
  13. Theorem \ref{['thm:Main']}: upper bound
  14. Theorem \ref{['thm:Main']}: lower bound
  15. Returning to the adversarial problem \ref{['Robust problem:Intro']}
  16. ...and 5 more sections

Key Result

Theorem 2.8

Suppose that Assumptionassump:CostStructure holds. Let $\mu$ be a finite positive measure over $\mathcal{Z}$. Then eqn:DualAdversarial is equivalent to the MOT problem Robust problem:Alternative_form with set of couplings $\Pi_K(\mu)$ defined as in eqn:CouplingsMOT, and cost function $\mathbf{c}$ de Furthermore, $def:RobustClassifPower=eqn:DualAdversarial$. In addition, from a solution pair $(\pi^

Figures (6)

  • Figure 1: Picture for \ref{['eq:generalized_barycenter']}. $\mu_i$'s are first moved to $\widetilde{\mu}_i$'s and $\lambda$ is chosen to cover all $\widetilde{\mu}_i$'s: it is the smallest positive measure which is larger than all $\widetilde{\mu}_i$'s.
  • Figure 2: (a) : Illustration of a partition of $\lambda$. (b) : Illustration of the transport from $\mu_{1, A}$'s to $\lambda_A$'s.
  • Figure 3: Picture for \ref{['eq:lambda_decomposed_formulation']}. Each of $\mu_{i,A}$'s is transported to $\lambda_A$ for all $i \in A$.
  • Figure 4: Illustrations of the adversarial attacks in all cases from section \ref{['ex : toy_example']}. Weights on arrows indicate the amount of mass the adversary moves to a perturbed point. $\overline{x}$'s are the support of $\lambda$ in \ref{['eq:generalized_barycenter']}. One observes that the support of $\lambda$ depends on both the geometry of data distributions and their magnitudes.
  • Figure 5: Three Gaussians in $\mathbb{R}^2$. One can observe that as $\varepsilon$ grows the robust classifying rule becomes simpler, as expected.
  • ...and 1 more figures

Theorems & Definitions (58)

  • Remark 2.2
  • Example 2.3
  • Remark 2.4
  • Example 2.5
  • Remark 2.6
  • Remark 2.7
  • Theorem 2.8
  • Proposition 3.1
  • Remark 3.2
  • Remark 3.3
  • ...and 48 more