Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TropNNC: Structured Neural Network Compression Using Tropical Geometry

Konstantinos Fotopoulos, Petros Maragos, Panagiotis Misiakos

TL;DR

TropNNC introduces a data-free, tropical-geometry-based framework for structured pruning of ReLU networks by representing layer outputs as tropical polynomials and rational functions, and compressing them via Hausdorff-distance-based zonotope approximation. The method extends tropical geometry to convolutional layers, tightens theoretical compression bounds, and provides both single- and multi-output algorithms, including iterative refinements and practical non-uniform variants. Empirically, TropNNC outperforms data-free baselines and matches or exceeds strong data-driven methods like ThiNet on MNIST, CIFAR, and ImageNet, with clear advantages in VGG-type architectures. The approach offers a scalable, principled path toward efficient, hardware-friendly neural network compression without retraining, underscoring the potential of tropical geometry for practical model reduction.

Abstract

We present TropNNC, a framework for compressing neural networks with linear and convolutional layers and ReLU activations using tropical geometry. By representing a network's output as a tropical rational function, TropNNC enables structured compression via reduction of the corresponding tropical polynomials. Our method refines the geometric approximation of previous work by adaptively selecting the weights of retained neurons. Key contributions include the first application of tropical geometry to convolutional layers and the tightest known theoretical compression bound. TropNNC requires only access to network weights - no training data - and achieves competitive performance on MNIST, CIFAR, and ImageNet, matching strong baselines such as ThiNet and CUP.

TropNNC: Structured Neural Network Compression Using Tropical Geometry

TL;DR

TropNNC introduces a data-free, tropical-geometry-based framework for structured pruning of ReLU networks by representing layer outputs as tropical polynomials and rational functions, and compressing them via Hausdorff-distance-based zonotope approximation. The method extends tropical geometry to convolutional layers, tightens theoretical compression bounds, and provides both single- and multi-output algorithms, including iterative refinements and practical non-uniform variants. Empirically, TropNNC outperforms data-free baselines and matches or exceeds strong data-driven methods like ThiNet on MNIST, CIFAR, and ImageNet, with clear advantages in VGG-type architectures. The approach offers a scalable, principled path toward efficient, hardware-friendly neural network compression without retraining, underscoring the potential of tropical geometry for practical model reduction.

Abstract

We present TropNNC, a framework for compressing neural networks with linear and convolutional layers and ReLU activations using tropical geometry. By representing a network's output as a tropical rational function, TropNNC enables structured compression via reduction of the corresponding tropical polynomials. Our method refines the geometric approximation of previous work by adaptively selecting the weights of retained neurons. Key contributions include the first application of tropical geometry to convolutional layers and the tightest known theoretical compression bound. TropNNC requires only access to network weights - no training data - and achieves competitive performance on MNIST, CIFAR, and ImageNet, matching strong baselines such as ThiNet and CUP.
Paper Structure (32 sections, 9 theorems, 81 equations, 8 figures, 4 tables, 3 algorithms)

This paper contains 32 sections, 9 theorems, 81 equations, 8 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Tropical Geometry of Neural Networks
  4. Neural Networks with Piecewise Linear Activations
  5. ReLU Activations.
  6. Zonotopes.
  7. Approximation based on Hausdorff distance
  8. Compression Algorithm
  9. Single output
  10. Algorithm for single output network.
  11. Multi-output
  12. Non-iterative Algorithm for multi-output network.
  13. Iterative Algorithm for multi-output network.
  14. Heuristic Improvements
  15. Deep and Convolutional Networks.
  16. ...and 17 more sections

Key Result

Theorem 2

Let $p, \tilde{p} \in \mathbb{R}_{\mathrm{max}}[\mathbf{x}]$ be two tropical polynomials with extended Newton polytopes $P = \mathrm{ENewt}(p)$ and $\tilde{P} = \mathrm{ENewt}(\tilde{p})$. Then, where $B = \{ \mathbf{x} \in \mathbb{R}^d : \| \mathbf{x} \| \leq r \}$ and $\rho = \sqrt{r^2 + 1}$.

Figures (8)

  • Figure 1: Neural network with one hidden ReLU layer. The first linear layer has weights $\{\mathbf{a}_i^T\}$ with bias $\{b_i\}$ corresponding to node $i\in [n]$ and the second has weights $\{c_{ji}\}$ between nodes $j \in [m], i \in [n]$
  • Figure 2: Example execution of TropNNC compared to the method of misiakos2022neural.
  • Figure 3: Example of simultaneous zonotope approximation for a network with 2 outputs and 2 hidden neurons
  • Figure 4: Compression of linear and convolutional layers of ReLU neural networks on MNIST datasets.
  • Figure 5: Compression of linear layers of AlexNet and convolutional layers of VGG on CIFAR datasets.
  • ...and 3 more figures

Theorems & Definitions (19)

  • Definition 1: Hausdorff distance
  • Theorem 2
  • Corollary 3
  • Proposition 4
  • Corollary 5
  • Example 1
  • Proposition 6
  • Proposition 7: zhang2018tropical
  • Lemma 8
  • proof
  • ...and 9 more