Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tropical Bisectors and Carlini-Wagner Attacks

Gillian Grindstaff, Julia Lindberg, Daniela Schkoda, Miruna-Stefana Sorea, Ruriko Yoshida

TL;DR

The combinatorics of tropical bisectors are explored and an upper bound on the number of linear segments the decision boundary of a tropical CNN can have is proved, and a refined version of the Carlini-Wagner attack is proposed, specifically tailored for the tropical architecture.

Abstract

Pasque et al. showed that using a tropical symmetric metric as an activation function in the last layer can improve the robustness of convolutional neural networks (CNNs) against state-of-the-art attacks, including the Carlini-Wagner attack. This improvement occurs when the attacks are not specifically adapted to the non-differentiability of the tropical layer. Moreover, they showed that the decision boundary of a tropical CNN is defined by tropical bisectors. In this paper, we explore the combinatorics of tropical bisectors and analyze how the tropical embedding layer enhances robustness against Carlini-Wagner attacks. We prove an upper bound on the number of linear segments the decision boundary of a tropical CNN can have. We then propose a refined version of the Carlini-Wagner attack, specifically tailored for the tropical architecture. Computational experiments with MNIST and LeNet5 showcase our attacks improved success rate.

Tropical Bisectors and Carlini-Wagner Attacks

TL;DR

The combinatorics of tropical bisectors are explored and an upper bound on the number of linear segments the decision boundary of a tropical CNN can have is proved, and a refined version of the Carlini-Wagner attack is proposed, specifically tailored for the tropical architecture.

Abstract

Pasque et al. showed that using a tropical symmetric metric as an activation function in the last layer can improve the robustness of convolutional neural networks (CNNs) against state-of-the-art attacks, including the Carlini-Wagner attack. This improvement occurs when the attacks are not specifically adapted to the non-differentiability of the tropical layer. Moreover, they showed that the decision boundary of a tropical CNN is defined by tropical bisectors. In this paper, we explore the combinatorics of tropical bisectors and analyze how the tropical embedding layer enhances robustness against Carlini-Wagner attacks. We prove an upper bound on the number of linear segments the decision boundary of a tropical CNN can have. We then propose a refined version of the Carlini-Wagner attack, specifically tailored for the tropical architecture. Computational experiments with MNIST and LeNet5 showcase our attacks improved success rate.

Paper Structure

This paper contains 15 sections, 10 theorems, 66 equations, 8 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Convolutional neural networks
  4. Tropical CNNs
  5. Combinatorics of tropical bisectors
  6. Computational Results
  7. Examples of tropical bisectors and their interaction with the gradient
  8. Carlini-Wagner Attacks
  9. Tropical CNNs
  10. CW attack
  11. Reason for robustness
  12. Improved attack by modified gradient descent
  13. Further improvement through multiple starting point gradient descent
  14. Appendix
  15. Networks considered

Key Result

Lemma 2.8

For any two points $x,u\in \mathbb R^{d+1} \!/\mathbb R {\bf 1}$, the gradient at $u$ of the tropical distance between $x$ and $u$ is given by if there are no ties in $(u-x)$, implying that the min- and max-sectors are uniquely identifiable, that is, the point $u$ is inside of open sectors and not on the boundary of $H_{-x}$.

Figures (8)

  • Figure 1: Tropical CNN architecture from Pasque2024.
  • Figure 2: Blue lines represent tropical hyperplanes $H_{-a}^{\max}$ and $H_{-b}^{\max}$ where $a = (0, 0, 0)$ and $b=(1, 2, 0)$. The red line represents the bisector ${\mathrm{bis}}(a, b)$.
  • Figure 3: The distribution of $\mathcal{C}(b)$ for 1000 independently sampled $b \in \mathbb{R}^d$, $b_i \sim \mathcal{N}(0,I)$. The experimental data achieves the upper bound from \ref{['lem:last-bound']} in a decreasing minority of cases as $d$ increases. We note that even when the upper bound is not realized, $\mathcal{C}(b)$ has a relatively small range of values for generic $b$.
  • Figure 4: Bisector of two points in the plane: $(0,0,0)$ and $(1,-1.5,0)$.
  • Figure 5: Bisector of two points in the plane: $(0,0,0)$ and $(1,2,0)$.
  • ...and 3 more figures

Theorems & Definitions (27)

  • Definition 2.1: Tropical Metric
  • Definition 2.2: Tropical Embedding Layer
  • Definition 2.3: Max-tropical Hyperplane ETC
  • Definition 2.4: Min-tropical Hyperplane ETC
  • Definition 2.5: Max-tropical Sectors from Section 5.5 in ETC
  • Definition 2.6: Min-tropical Sectors
  • Definition 2.7: Indicators for Open Sectors
  • Lemma 2.8: BSYM
  • Lemma 2.9: miranda2017softmax
  • Theorem 2.10: BSYM
  • ...and 17 more