A Tiny Transformer for Low-Power Arrhythmia Classification on Microcontrollers

TL;DR

This work presents a tiny Vision-Transformer–based ECG arrhythmia classifier tailored for microcontroller deployment, achieving 99.05% intra-patient accuracy on MIT-BIH benchmarks with a storage footprint around 128 kB and only about 6k parameters. The model integrates a 1D convolutional embedding, an encoder with Multi-Head Attention, and an RR-interval input, achieving 8-bit quantization with minimal accuracy loss (≈0.1%) and robust performance under electrode motion artifacts via noise-augmented training. Post-deployment experiments on the GAP9 MCU demonstrate real-time inference (~2.85–4.28 ms) and low energy consumption (≈0.09–0.12 mJ), confirming practicality for wearable, always-on arrhythmia monitoring. The results show that transformer-based approaches can be efficiently scaled to edge devices while preserving high diagnostic accuracy, enabling continuous, privacy-preserving cardiac monitoring at the sensor level.

Abstract

Wearable systems for the continuous and real-time monitoring of cardiovascular diseases are becoming widespread and valuable assets in diagnosis and therapy. A promising approach for real-time analysis of the electrocardiographic (ECG) signal and the detection of heart conditions, such as arrhythmia, is represented by the transformer machine learning model. Transformers are powerful models for the classification of time series, although efficient implementation in the wearable domain raises significant design challenges, to combine adequate accuracy and a suitable complexity. In this work, we present a tiny transformer model for the analysis of the ECG signal, requiring only 6k parameters and reaching 98.97% accuracy in the recognition of the 5 most common arrhythmia classes from the MIT-BIH Arrhythmia database, assessed considering 8-bit integer inference as required for efficient execution on low-power microcontroller-based devices. We explored an augmentation-based training approach for improving the robustness against electrode motion artifacts noise, resulting in a worst-case post-deployment performance assessment of 98.36% accuracy. Suitability for wearable monitoring solutions is finally demonstrated through efficient deployment on the parallel ultra-low-power GAP9 processor, where inference execution requires 4.28ms and 0.09mJ.

Figures (6)

• Figure 1: Typical ECG signal. Highlighted is the considered window around a single heartbeat, the R peak position and the definition considered in the following for pre- and post- RR intervals.
• Figure 2: Overview of the ECG monitoring system.
• Figure 3: Architecture of the proposed transformer model for arrhythmia classification. The shape of the weight tensor in each layer is indicated with the prefix W, whereas the prefix in refers to the shape of the input data tensors.
• Figure 4: Exploration of the architectural parameters based on left-out test accuracy on the MIT-BIH dataset. Each dot in the plot represents one of the evaluated design points, resulting from the selection of the parameters explored: embedding size, kernel size, number of heads, and hidden layer size. Dot placement in a) considers the memory footprint of the model, accounting for the storage of the partial results produced during inference computation, whereas placement in b) considers the number of parameters in the model. The selected point is highlighted in black. Shaded grey area: outside of the 128 kB feasibility region.
• Figure 5: Test classification performance considering increasing levels of noise deriving from electrode motion artifacts.