On-device Online Learning and Semantic Management of TinyML Systems

TL;DR

The paper addresses the gap between prototype TinyML models and production-ready on-device systems by tackling concept drift, deployment heterogeneity, and resource management. It combines TinyOL for on-device online learning, TinyReptile and TinyMetaFed for federated meta-learning to enable rapid adaptation across devices, and SeLoC-ML for semantic, scalable management of TinyML resources, including low-code deployment via Mendix. Key contributions include a compact on-device training layer, federated meta-learning frameworks with communication/privacy optimizations, and a semantic knowledge-graph approach to model-device matchmaking and code generation. Empirical results across handwritten character classification, keyword spotting, and smart-building presence detection demonstrate improved adaptability, reduced communication and engineering effort, and practical viability for industrial TinyML deployments.

Abstract

Recent advances in Tiny Machine Learning (TinyML) empower low-footprint embedded devices for real-time on-device Machine Learning. While many acknowledge the potential benefits of TinyML, its practical implementation presents unique challenges. This study aims to bridge the gap between prototyping single TinyML models and developing reliable TinyML systems in production: (1) Embedded devices operate in dynamically changing conditions. Existing TinyML solutions primarily focus on inference, with models trained offline on powerful machines and deployed as static objects. However, static models may underperform in the real world due to evolving input data distributions. We propose online learning to enable training on constrained devices, adapting local models towards the latest field conditions. (2) Nevertheless, current on-device learning methods struggle with heterogeneous deployment conditions and the scarcity of labeled data when applied across numerous devices. We introduce federated meta-learning incorporating online learning to enhance model generalization, facilitating rapid learning. This approach ensures optimal performance among distributed devices by knowledge sharing. (3) Moreover, TinyML's pivotal advantage is widespread adoption. Embedded devices and TinyML models prioritize extreme efficiency, leading to diverse characteristics ranging from memory and sensors to model architectures. Given their diversity and non-standardized representations, managing these resources becomes challenging as TinyML systems scale up. We present semantic management for the joint management of models and devices at scale. We demonstrate our methods through a basic regression example and then assess them in three real-world TinyML applications: handwritten character image classification, keyword audio classification, and smart building presence detection, confirming our approaches' effectiveness.

Figures (18)

• Figure 1: A typical workflow for the development of a TinyML model.
• Figure 2: The pipeline for developing TinyML in industrial settings. This work focuses on the previously overlooked components marked in orange.
• Figure 3: Five random sine wave functions for five TinyML devices.
• Figure 4: Building blocks of TinyOL.
• Figure 5: Demonstration of TinyOL using the sine wave regression example: Suppose the data distribution between the training and testing data, labeled as "Sine wave - training data" and "Sine wave - testing data," differs due to changing real-world conditions. TinyOL helps the NN improve its performance after deployment by enabling on-device post-learning. The results on the testing data before and after TinyOL enabled are marked as "Validation on testing data before TinyOL" and "Validation on testing data after TinyOL." We can observe that after fine-tuning, the predicted output aligns much better with the distribution of the testing data.