TActiLE: Tiny Active LEarning for wearable devices
Massimo Pavan, Claudio Galimberti, Manuel Roveri
TL;DR
This paper addresses the labeling bottleneck of On-device Learning for TinyML wearables by introducing TActiLE, a streaming Active Learning framework tailored for resource-constrained devices. It presents two batch-handling strategies, Info RV (uncertainty-based) and Dual RV (uncertainty plus diversity), combined with a Batch Reaching trigger, and demonstrates substantial accuracy gains over baselines on MNIST, Fashion-MNIST, and CIFAR-10 under realistic TinyML constraints. The results indicate that on limited hardware, Info RV excels, while Dual RV offers advantages for more complex data, with favorable trade-offs in time and memory. Overall, the work advances practical on-device AL for wearables, enabling personalized, privacy-preserving learning directly on-device and opening avenues for broader data modalities and richer selection metrics.
Abstract
Tiny Machine Learning (TinyML) algorithms have seen extensive use in recent years, enabling wearable devices to be not only connected but also genuinely intelligent by running machine learning (ML) computations directly on-device. Among such devices, smart glasses have particularly benefited from TinyML advancements. TinyML facilitates the on-device execution of the inference phase of ML algorithms on embedded and wearable devices, and more recently, it has expanded into On-device Learning (ODL), which allows both inference and learning phases to occur directly on the device. The application of ODL techniques to wearable devices is particularly compelling, as it enables the development of more personalized models that adapt based on the data of the user. However, one of the major challenges of ODL algorithms is the scarcity of labeled data collected on-device. In smart wearable contexts, requiring users to manually label large amounts of data is often impractical and could lead to user disengagement with the technology. To address this issue, this paper explores the application of Active Learning (AL) techniques, i.e., techniques that aim at minimizing the labeling effort, by actively selecting from a large quantity of unlabeled data only a small subset to be labeled and added to the training set of the algorithm. In particular, we propose TActiLE, a novel AL algorithm that selects from the stream of on-device sensor data the ones that would help the ML algorithm improve the most once coupled with labels provided by the user. TActiLE is the first Active Learning technique specifically designed for the TinyML context. We evaluate its effectiveness and efficiency through experiments on multiple image classification datasets. The results demonstrate its suitability for tiny and wearable devices.