RockNet: Distributed Learning on Ultra-Low-Power Devices

TL;DR

RockNet addresses the challenge of training high-accuracy models on ultra-low-power CPS devices by formulating a split TinyML approach that distributes ROCKET-based timeseries classifiers across a multi-hop wireless network using the Mixer protocol. The method achieves state-of-the-art accuracy without offline pretraining and significantly reduces per-device memory, latency, and energy as the device count grows, validated on a 20-device BLE testbed. By combining input-partitioned ROCKET with efficient all-to-all synchronization, RockNet demonstrates feasible online learning on constrained hardware and highlights practical trade-offs between computation and communication. This work unlocks high-accuracy, on-device training for CPS, enabling privacy-preserving and low-latency adaptation in real-world deployments.

Abstract

As Machine Learning (ML) becomes integral to Cyber-Physical Systems (CPS), there is growing interest in shifting training from traditional cloud-based to on-device processing (TinyML), for example, due to privacy and latency concerns. However, CPS often comprise ultra-low-power microcontrollers, whose limited compute resources make training challenging. This paper presents RockNet, a new TinyML method tailored for ultra-low-power hardware that achieves state-of-the-art accuracy in timeseries classification, such as fault or malware detection, without requiring offline pretraining. By leveraging that CPS consist of multiple devices, we design a distributed learning method that integrates ML and wireless communication. RockNet leverages all devices for distributed training of specialized compute efficient classifiers that need minimal communication overhead for parallelization. Combined with tailored and efficient wireless multi-hop communication protocols, our approach overcomes the communication bottleneck that often occurs in distributed learning. Hardware experiments on a testbed with 20 ultra-low-power devices demonstrate RockNet's effectiveness. It successfully learns timeseries classification tasks from scratch, surpassing the accuracy of the latest approach for neural network microcontroller training by up to 2x. RockNet's distributed ML architecture reduces memory, latency and energy consumption per device by up to 90 % when scaling from one central device to 20 devices. Our results show that a tight integration of distributed ML, distributed computing, and communication enables, for the first time, training on ultra-low-power hardware with state-of-the-art accuracy.

Figures (10)

• Figure 1: Central versus Federated versus Split TinyML. Central TinyML: All data is sent to one device that runs the ML algorithm. Federated TinyML: Each device trains a model on its local data; these models are repeatedly shared and merged. Split TinyML: Devices collaboratively run the ML algorithm, each holding portions of the parameters $W$.
• Figure 2: Overview of RockNet. During the forward pass (feature extraction + linear classifier), all devices exchange their data with each other. The training step does not require any additional communication, making RockNet more efficient.
• Figure 3: Pipelining of RockNet. In our implementation, we process three timeseries simultaneously to reduce the overhead induced by communication.
• Figure 4: Overview of our testbed. It consists of up to 20 ultra-low-power Bluetooth radios forming a two-hop network.
• Figure 5: Accuracy of RockNet versus central NN training with AIfES. RockNet achieves a higher accuracy than AIfES in more than 78% of the runs with 32 ADAM and in 67% of the runs with 8 ADAM.