An Optimal Transport Approach for Computing Adversarial Training Lower Bounds in Multiclass Classification

TL;DR

This work formulates the computation of classifier-agnostic lower bounds for the optimal adversarial risk in multiclass classification as a multimarginal OT problem and relies on a tractable truncation of higher-order class interactions. It provides two practical algorithms—LP and entropic regularization (Sinkhorn)—to solve the truncated MOT, with rigorous complexity bounds and a rounding scheme to maintain feasibility. Empirical results on MNIST and CIFAR-$10$ show that truncation (up to order 2–3) yields tight lower bounds while substantially reducing computational cost, confirming the approach's practicality for realistic, large-class problems. The methods enable efficient, universal benchmarks for robustness in multiclass learning and pave the way for scalable DRO analyses in complex settings.

Abstract

Despite the success of deep learning-based algorithms, it is widely known that neural networks may fail to be robust. A popular paradigm to enforce robustness is adversarial training (AT), however, this introduces many computational and theoretical difficulties. Recent works have developed a connection between AT in the multiclass classification setting and multimarginal optimal transport (MOT), unlocking a new set of tools to study this problem. In this paper, we leverage the MOT connection to propose computationally tractable numerical algorithms for computing universal lower bounds on the optimal adversarial risk and identifying optimal classifiers. We propose two main algorithms based on linear programming (LP) and entropic regularization (Sinkhorn). Our key insight is that one can harmlessly truncate the higher order interactions between classes, preventing the combinatorial run times typically encountered in MOT problems. We validate these results with experiments on MNIST and CIFAR-$10$, which demonstrate the tractability of our approach.

Figures (8)

• Figure 1: (Top) A simple six class dataset with 50 points in each class. Filled regions are colored according to the class of the nearest point. (Bottom) The optimal adversarial attack applied to the dataset on the left. The shared colors from the left figure represent the singleton classes $\mu_{\{1\}},...,\mu_{\{6\}}$ and the blended colors represent the various $\mu_{A}$ for $|A| \geq 2$. Points are colored according to the combination of classes they are associated with. Filled regions are colored according to the combination of the nearest point.
• Figure 2: Plots of the value of \ref{['eq : truncation of stratified MOT']} and the upper bound provided by Proposition \ref{['prop : approximation bounds']} for a range of settings of $\varepsilon$ and $K = 2,3$ as well as the untruncated values. These are derived from synthetic data using 20 samples from six classes.
• Figure 3: Lower bound of adversarial risk of and runtimes of the entropic regularization and LP for MNIST and CIFAR-10. The left plots and the right ones are equipped with $\ell^2$ norm and $\ell^{\infty}$ norm, respectively. For LP with truncation up to 3, due to the huge complexity we stop the computing earlier.
• Figure 4: The optimal adversarial risks for MNIST and CIFAR-10 with 4 classes.
• Figure 5: Contribution by interactions of each order to the optimal multicoupling as the budge $\epsilon$ varies in three different settings.

Theorems & Definitions (31)

• Proposition 3.1
• Definition 3.2
• Proposition 3.3
• proof
• Remark 3.4
• Remark 3.5
• Remark 3.6
• Remark 3.7
• Theorem 3.8
• Remark 3.9