Compound and Parallel Modes of Tropical Convolutional Neural Networks

TL;DR

This work tackles the computational bottleneck of deep CNNs by introducing tropical convolutional variants that replace multiplications with min-plus and max-plus operations. Building on TCNN, the paper proposes compound (cTCNN) and parallel (pTCNN) modes that blend tropical kernels, with extensions to 1D and 3D data and a PyTorch-compatible tcnn framework. Through extensive experiments on 1D/2D/3D tasks and deeper networks like ResNet variants, it demonstrates that cTCNN and pTCNN can match or exceed traditional CNN performance while substantially reducing multiplications, and that integrating these blocks into deeper architectures yields further gains. The study also explores efficient, simplified TCNN structures suitable for resource-constrained settings, highlighting practical implications for edge deployment and hardware acceleration of tropical neural networks.

Abstract

Convolutional neural networks have become increasingly deep and complex, leading to higher computational costs. While tropical convolutional neural networks (TCNNs) reduce multiplications, they underperform compared to standard CNNs. To address this, we propose two new variants - compound TCNN (cTCNN) and parallel TCNN (pTCNN)-that use combinations of tropical min-plus and max-plus kernels to replace traditional convolution kernels. This reduces multiplications and balances efficiency with performance. Experiments on various datasets show that cTCNN and pTCNN match or exceed the performance of other CNN methods. Combining these with conventional CNNs in deeper architectures also improves performance. We are further exploring simplified TCNN architectures that reduce parameters and multiplications with minimal accuracy loss, aiming for efficient and effective models.

Figures (5)

• Figure 1: Samples of the UrbanSound8K dataset
• Figure 2: Samples of the MNIST dataset and the CIFAR-10 dataset
• Figure 3: Overview of the MedMNIST v2 dataset
• Figure 4: Diagram of Different Basic Blocks (a) Basic block: Contains a convolutional layer (Conv), batch normalization layer (BN), ReLU activation function, another convolutional layer, and batch normalization layer. (b) Compound basic block 1: Contains a convolutional layer, batch normalization layer, compound convolutional layer (CompoundConv), and batch normalization layer. (c) Compound basic block 2: Contains two compound convolutional layers and batch normalization layers. (d) Parallel basic block 1: Contains a convolutional layer, batch normalization layer, parallel convolutional layer (ParallelConv), and batch normalization layer. (e) Parallel basic block 2: Contains two parallel convolutional layers and batch normalization layers.
• Figure 5: Hierarchical Design of the Tropical Convolution Neural Networks Framework: The framework supports tropical convolution across various applications, including time series prediction, speech recognition, and image classification. The framework is organized into multiple layers, encompassing Tconv layers, operators, and class definitions, all built on top of the PyTorch library. It supports diverse hardware backends such as CPU, GPU, TPU, and FPGA. The Tconv layers implement a variety of operations, including MinPlusSumConv and CompoundMinMaxPlusSumConv, optimized for different input dimensions (1D, 2D, 3D). Operators such as combine_sum, min, max, and preprocess functions are provided to facilitate the construction of complex neural network architectures tailored to different applications.

Theorems & Definitions (3)

• Definition 1
• Definition 2
• Definition 3