Fast Data Aware Neural Architecture Search via Supernet Accelerated Evaluation
Emil Njor, Colby Banbury, Xenofon Fafoutis
TL;DR
This work tackles the challenge of deploying TinyML on ultra-low-power hardware by extending hardware-aware neural architecture search to data configurations, creating Data Aware Neural Architecture Search (DNAS). The authors introduce a supernet-based evaluation framework built on a MobileNetV2 backbone to efficiently search across both input data configurations (e.g., resolution and color encoding) and model architectures, enabling tractable optimization on the Wake Vision dataset. Results show that DNAS consistently yields superior TinyML systems under the same resource constraints and that the supernet-based approach dramatically speeds up the search while achieving higher accuracy ceilings than prior non-supernet methods. The work demonstrates practical gains in resource efficiency and predictive performance, offering a scalable pathway for automating TinyML deployment with reduced expert intervention and broader hardware applicability.
Abstract
Tiny machine learning (TinyML) promises to revolutionize fields such as healthcare, environmental monitoring, and industrial maintenance by running machine learning models on low-power embedded systems. However, the complex optimizations required for successful TinyML deployment continue to impede its widespread adoption. A promising route to simplifying TinyML is through automatic machine learning (AutoML), which can distill elaborate optimization workflows into accessible key decisions. Notably, Hardware Aware Neural Architecture Searches - where a computer searches for an optimal TinyML model based on predictive performance and hardware metrics - have gained significant traction, producing some of today's most widely used TinyML models. Nevertheless, limiting optimization solely to neural network architectures can prove insufficient. Because TinyML systems must operate under extremely tight resource constraints, the choice of input data configuration, such as resolution or sampling rate, also profoundly impacts overall system efficiency. Achieving truly optimal TinyML systems thus requires jointly tuning both input data and model architecture. Despite its importance, this "Data Aware Neural Architecture Search" remains underexplored. To address this gap, we propose a new state-of-the-art Data Aware Neural Architecture Search technique and demonstrate its effectiveness on the novel TinyML ``Wake Vision'' dataset. Our experiments show that across varying time and hardware constraints, Data Aware Neural Architecture Search consistently discovers superior TinyML systems compared to purely architecture-focused methods, underscoring the critical role of data-aware optimization in advancing TinyML.