Genetic Column Generation for Computing Lower Bounds for Adversarial Classification

TL;DR

This work investigates how ideas from Genetic Column Generation for multi-marginal optimal transport can be used to overcome the curse of dimension in computing the minimal adversarial risk in multi-class classification.

Abstract

Recent theoretical results on adversarial multi-class classification showed a similarity to the multi-marginal formulation of Wasserstein-barycenter in optimal transport. Unfortunately, both problems suffer from the curse of dimension, making it hard to exploit the nice linear program structure of the problems for numerical calculations. We investigate how ideas from Genetic Column Generation for multi-marginal optimal transport can be used to overcome the curse of dimension in computing the minimal adversarial risk in multi-class classification.

Figures (7)

• Figure 1: A synthetic data set of 10 overlapping Gaussian distributions. The proximity of the clusters limits the classification power and makes it a hard classification problem.
• Figure 2: The figure shows the number of configurations per adversarial budget in log-scale. Each colored line indicates configurations of a certain length. The data set features 10 classes, limiting the maximal length of a configuration. The corresponding search times are reported in Table \ref{['tab:TwoDim-nconf']}.
• Figure 3: Convergence of the genetic search rule for the classical $W_\infty$-regularization. For small budgets, an optimal set of configurations is quickly found. The upper figure shows the optimal cost of the reduced problem in dependence on computation time; the lower figure shows the difference to the true optimal cost found by the exhaustive search in logarithmic scale. For larger budgets, the optimal cost often stagnates, but as seen for budget $\varepsilon = 0.22$ it is possible to find an optimal set of configurations.
• Figure 4: Minimal adversarial risk for the classification problem for the Euclidean metric. The x-axis indicates the adversarial budget $\varepsilon$, and the y-axis indicates the corresponding minimal adversarial risk. The relative error of the genetic search rule compared to the exhaustive search is below 1%.
• Figure 5: Left: Convergence of Algorithm \ref{['alg:w2algo']} for the synthetic data set. The convergence speed decreases with increasing $\tau$. The lower plot shows the relative optimal cost of the reduced problem on a logarithmic scale with base 10 to visualize the convergence speed. Right: The optimal values of Problem \ref{['eq:w2penalized']} for different regularization strengths $\tau \in [0,6]$. For $\tau \geq 5.0$ the corrected adversarial risk is the maximal adversarial risk ($1-\frac{119}{1000}$) for that data set.

Theorems & Definitions (5)

• Proposition
• Lemma 1
• proof
• Lemma 2
• proof