Natively neuromorphic LMU architecture for encoding-free SNN-based HAR on commercial edge devices

TL;DR

This work addresses the bottleneck of interfacing neuromorphic computation with non-dedicated edge hardware by introducing L2MU, a natively neuromorphic Legendre Memory Unit built entirely from Leaky Integrate-and-Fire populations that can process raw sensor data without input encoding. By translating LMU blocks into spike-based neuron populations and coupling them with a three-layer encoding module, the authors demonstrate encoding-free RSNN capabilities for Human Activity Recognition on wearable device data. Across training, compression, and hardware deployment, Leaky neuron variants achieve higher accuracy and robustness than Synaptic variants, while pruning and fine-tuning substantially reduce memory footprint and synaptic operations with minimal or improving accuracy. Deployment on STM32 and Raspberry Pi platforms (via ONNX Runtime) achieves real-time inference (under 300 ms) with energy consumption in the tens to hundreds of millijoules per inference, illustrating the practical viability of neuromorphic approaches on common edge devices.

Abstract

Neuromorphic models take inspiration from the human brain by adopting bio-plausible neuron models to build alternatives to traditional Machine Learning (ML) and Deep Learning (DL) solutions. The scarce availability of dedicated hardware able to actualize the emulation of brain-inspired computation, which is otherwise only simulated, yet still hinders the wide adoption of neuromorphic computing for edge devices and embedded systems. With this premise, we adopt the perspective of neuromorphic computing for conventional hardware and we present the L2MU, a natively neuromorphic Legendre Memory Unit (LMU) which entirely relies on Leaky Integrate-and-Fire (LIF) neurons. Specifically, the original recurrent architecture of LMU has been redesigned by modelling every constituent element with neural populations made of LIF or Current-Based (CuBa) LIF neurons. To couple neuromorphic computing and off-the-shelf edge devices, we equipped the L2MU with an input module for the conversion of real values into spikes, which makes it an encoding-free implementation of a Recurrent Spiking Neural Network (RSNN) able to directly work with raw sensor signals on non-dedicated hardware. As a use case to validate our network, we selected the task of Human Activity Recognition (HAR). We benchmarked our L2MU on smartwatch signals from hand-oriented activities, deploying it on three different commercial edge devices in compressed versions too. The reported results remark the possibility of considering neuromorphic models not only in an exclusive relationship with dedicated hardware but also as a suitable choice to work with common sensors and devices.

Figures (6)

• Figure 1: Schematic of the whole model. The 6-axis input is shown as six different channels in the leftmost part. Connected to them is the 3-layer, fully-connected, encoding module. It transmits spikes to the which in turn feeds the output layer with the spikes from the hidden state (h) population.
• Figure 2: Learning curves from retraining of the best model with Leaky neurons. The accuracy for the optimal model identified through (a) is reported together with the results for the compressed model (b). Mean values and standard deviations are shown by solid line and shaded area respectively.
• Figure 3: Learning curves from retraining of the best model with Synaptic neurons. The accuracy for the optimal model identified through (a) is reported together with the results for the compressed model (b). Mean values and standard deviations are shown by solid line and shaded area respectively.
• Figure 4: Illustrative confusion matrices for a detailed breakdown of the classification performance achieved by the optimized models after retraining with ten different seed values. With Leaky neurons (a), a maximum test accuracy of 94.14% is achieved; while 93.48% is the highest result obtained on test data with Synaptic neurons (b).
• Figure 5: Comparison of the different models. Results from the median values of the statistical analyses are shown. With both Leaky and Synaptic neurons, an increase in test accuracy is achieved performing pruning and fine-tuning. On the contrary, the memory footprint and the number of synaptic operations per sample are reduced, for both neuron types, by the compressed models.