Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tropical Decision Boundaries for Neural Networks Are Robust Against Adversarial Attacks

Kurt Pasque, Christopher Teska, Ruriko Yoshida, Keiji Miura, Jefferson Huang

TL;DR

Neural networks remain vulnerable to adversarial perturbations, motivating defenses that preserve accuracy without excessive computation. The authors propose a tropical CNN that embeds inputs in the tropical projective torus and uses a tropical last-layer decision boundary, where each class is represented by a Fermat-Weber point and scores are $d_{tr}$ distances followed by a softmin. They provide a geometric analysis showing the decision boundary of the tropical network lies on tropical balls and their intersections, and validate robustness across MNIST, SVHN, and CIFAR-10 against attacks such as PGD, CW, and SPSA, while maintaining competitive accuracy and training efficiency. Overall, the approach offers a principled, low-overhead path to adversarial robustness by leveraging max-plus algebra and tropical geometry to shape decision regions.

Abstract

We introduce a simple, easy to implement, and computationally efficient tropical convolutional neural network architecture that is robust against adversarial attacks. We exploit the tropical nature of piece-wise linear neural networks by embedding the data in the tropical projective torus in a single hidden layer which can be added to any model. We study the geometry of its decision boundary theoretically and show its robustness against adversarial attacks on image datasets using computational experiments.

Tropical Decision Boundaries for Neural Networks Are Robust Against Adversarial Attacks

TL;DR

Neural networks remain vulnerable to adversarial perturbations, motivating defenses that preserve accuracy without excessive computation. The authors propose a tropical CNN that embeds inputs in the tropical projective torus and uses a tropical last-layer decision boundary, where each class is represented by a Fermat-Weber point and scores are distances followed by a softmin. They provide a geometric analysis showing the decision boundary of the tropical network lies on tropical balls and their intersections, and validate robustness across MNIST, SVHN, and CIFAR-10 against attacks such as PGD, CW, and SPSA, while maintaining competitive accuracy and training efficiency. Overall, the approach offers a principled, low-overhead path to adversarial robustness by leveraging max-plus algebra and tropical geometry to shape decision regions.

Abstract

We introduce a simple, easy to implement, and computationally efficient tropical convolutional neural network architecture that is robust against adversarial attacks. We exploit the tropical nature of piece-wise linear neural networks by embedding the data in the tropical projective torus in a single hidden layer which can be added to any model. We study the geometry of its decision boundary theoretically and show its robustness against adversarial attacks on image datasets using computational experiments.
Paper Structure (21 sections, 14 theorems, 54 equations, 7 figures, 9 tables)

This paper contains 21 sections, 14 theorems, 54 equations, 7 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Basics in Tropical Geometry and Tropical Bisectors
  3. Tropical Convolutional Neural Networks
  4. Tropical Embedding Layer
  5. Structure of a Tropical Convolutional Neural Network
  6. Definition and Analysis of Tropical Decision Boundaries
  7. Decision Boundary of a ReLU Neural Network
  8. Decision Boundary of a Tropical Neural Network
  9. Computational Experiments
  10. Results
  11. Decision Boundaries Visualized
  12. Discussion
  13. Adversarial Attacks on Classifiers
  14. Defenses Against Adversarial Attacks
  15. Conclusion
  16. ...and 6 more sections

Key Result

Theorem 12

A tropical ball $B_{x}(r)$ is a polytrope, which is a tropical simplex (and hence a tropical polytope) that is also a classical polytope.

Figures (7)

  • Figure 1: Here we have $w^*_{1,1} = (5, -5, 0), \, w^*_{2,1} = (-5, 5, 0)$. (LEFT) Heat-map plot for distance from optimal weights in Example \ref{['eg:contour']}. The white line is the decision boundary. (RIGHT) Contour plot of Example \ref{['eg:contour']}. As you can see tropical balls $B_{-w^*_{1,1}}(r) = B_{(5, -5)}(r)$ and $B_{-w^*_{2,1}}(r) = B_{(-5, 5)}(r)$ for $r > 0$.
  • Figure 2: Here we have $w^*_{1,1} = (-2, -7, 0), \, w^*_{1,2} = (-8, -3, 0), \, w^*_{2,1} = (6, 1, 0), \, w^*_{2,2} = (0, 5, 0)$. (LEFT) Heat-map plot for distance from optimal weights in Example \ref{['eg:contour']}. The white line is the decision boundary. (RIGHT) Contour plot of Example \ref{['eg:contour2']}. As you can see tropical balls $B_{-w^*_{1,1}}(r) = B_{(2, 7)}(r), B_{-w^*_{1,2}}(r) = B_{(8, 3)}(r)$ and $B_{-w^*_{2,1}}(r) = B_{(-6, -1)}(r), B_{-w^*_{2,2}}(r) = B_{(0, -5)}(r)$ for $r > 0$.
  • Figure 3: Here we have $w^*_{1,1} = (5, 5, 0), \, w^*_{2,1} = (-7, -7, 0)$. (LEFT) Heat-map plot for distance from optimal weights in Example \ref{['eg:contourNonFull']}. (RIGHT) Contour plot of Example \ref{['eg:contourNonFull']}.
  • Figure 4: The bisectors between $w^*_{1, 1}=(0, 0, 0)$ and $w^*_{2, 1} = (1, w, 0)$ for various $w$ are represented by red. The lightgray lines represent the hyperplanes for $w^*_{1, 1}$ and $w^*_{2, 1}$. Note that $w=1$ and $w=0$ are not in weakly general positions (i. e. $w^*_{2, 1} - w^*_{1, 1}$ is parallel to a facet of a tropical unit ball) and, therefore, not in general positions (i. e. small perturbations change which sectors the bisector is in). $w=-1$ is in weakly general positions but not in general positions.
  • Figure 5: Decision boundaries MNIST-trained example.
  • ...and 2 more figures

Theorems & Definitions (44)

  • Definition 1: Tropical Arithmetic Operations
  • Definition 2: Tropical Metric
  • Remark 3
  • Definition 4: Tropical Ball
  • Remark 5
  • Definition 6: Tropical Bisector
  • Definition 7: Definition 1 in CJS
  • Example 8
  • Definition 9: Tropical Embedding Layer
  • Remark 10
  • ...and 34 more