TinyTNAS: GPU-Free, Time-Bound, Hardware-Aware Neural Architecture Search for TinyML Time Series Classification

TL;DR

TinyTNAS is presented, a novel hardware-aware multi-objective Neural Architecture Search (NAS) tool specifically designed for TinyML time series classification that operates efficiently on CPUs, making it accessible for a broader range of applications.

Abstract

In this work, we present TinyTNAS, a novel hardware-aware multi-objective Neural Architecture Search (NAS) tool specifically designed for TinyML time series classification. Unlike traditional NAS methods that rely on GPU capabilities, TinyTNAS operates efficiently on CPUs, making it accessible for a broader range of applications. Users can define constraints on RAM, FLASH, and MAC operations to discover optimal neural network architectures within these parameters. Additionally, the tool allows for time-bound searches, ensuring the best possible model is found within a user-specified duration. By experimenting with benchmark dataset UCI HAR, PAMAP2, WISDM, MIT BIH, and PTB Diagnostic ECG Databas TinyTNAS demonstrates state-of-the-art accuracy with significant reductions in RAM, FLASH, MAC usage, and latency. For example, on the UCI HAR dataset, TinyTNAS achieves a 12x reduction in RAM usage, a 144x reduction in MAC operations, and a 78x reduction in FLASH memory while maintaining superior accuracy and reducing latency by 149x. Similarly, on the PAMAP2 and WISDM datasets, it achieves a 6x reduction in RAM usage, a 40x reduction in MAC operations, an 83x reduction in FLASH, and a 67x reduction in latency, all while maintaining superior accuracy. Notably, the search process completes within 10 minutes in a CPU environment. These results highlight TinyTNAS's capability to optimize neural network architectures effectively for resource-constrained TinyML applications, ensuring both efficiency and high performance. The code for TinyTNAS is available at the GitHub repository and can be accessed at https://github.com/BidyutSaha/TinyTNAS.git.

Figures (3)

• Figure 1: Concept Diagram of TinyTNAS
• Figure 2: Graphical representation of candidate architectures created by TinyTNAS, where $k$ denotes the number of filters in the first depthwise separable 1D convolutional layer and $c$ represents the number of repeating blocks. The kernel size of each depthwise separable 1D CNN is 3 with a stride of 1 and ReLU activation. The final dense layer uses a softmax activation function, while the preceding dense layer uses a ReLU activation function.
• Figure 3: Architectures Generated by TinyTNAS under Specific Constraints on Various Datasets. Constraints include maximum RAM of 20 KB, maximum FLASH of 64 KB, maximum MAC of 60K, and a maximum search time of 10 minutes. DSC1 denotes Depthwise Separable 1D Convolution with a kernel size of 3 and ReLU activation. MP1 represents Max Pooling 1D with a size of 2. GAP1 indicates Global Average Pooling 1D. DR refers to a Dense Layer with ReLU activation, and DS denotes a Dense Layer with Softmax activation.