Decoding finger velocity from cortical spike trains with recurrent spiking neural networks

TL;DR

This work investigates fully implanted, ultra-low-power decoding of finger velocity from cortical spike trains using recurrent spiking neural networks. By building a large bigRSNN and a compact tinyRSNN, the authors show that RSNNs can achieve competitive decoding performance while dramatically reducing memory and energy requirements, thanks to activity regularization and iterative pruning. The bigRSNN delivers the highest accuracy, whereas tinyRSNN demonstrates a strong accuracy- and energy-efficiency Pareto frontier, reducing energy by about 26.6% versus a SNN baseline and by roughly 500× versus an ANN baseline. These results suggest RSNN-based decoders are viable candidates for fully implanted brain-machine interfaces with stringent latency and power constraints, potentially transforming patient care through ultra-low-power neuromorphic implementations.

Abstract

Invasive cortical brain-machine interfaces (BMIs) can significantly improve the life quality of motor-impaired patients. Nonetheless, externally mounted pedestals pose an infection risk, which calls for fully implanted systems. Such systems, however, must meet strict latency and energy constraints while providing reliable decoding performance. While recurrent spiking neural networks (RSNNs) are ideally suited for ultra-low-power, low-latency processing on neuromorphic hardware, it is unclear whether they meet the above requirements. To address this question, we trained RSNNs to decode finger velocity from cortical spike trains (CSTs) of two macaque monkeys. First, we found that a large RSNN model outperformed existing feedforward spiking neural networks (SNNs) and artificial neural networks (ANNs) in terms of their decoding accuracy. We next developed a tiny RSNN with a smaller memory footprint, low firing rates, and sparse connectivity. Despite its reduced computational requirements, the resulting model performed substantially better than existing SNN and ANN decoders. Our results thus demonstrate that RSNNs offer competitive CST decoding performance under tight resource constraints and are promising candidates for fully implanted ultra-low-power BMIs with the potential to revolutionize patient care.

Figures (2)

• Figure 1: Setup and example network activity.A) Schematic of the RSNN network architecture. B) The publicly available dataset consists of CST recorded from two macaque monkeys performing a self-paced reaching task using chronically implanted electrodes. Recording sites were either in the primary motor cortex (M1) (Indy) or in M1 and the primary sensory cortex (S1) (Loco) odoherty_nonhuman_2017. C) Example network activity for tinyRSNN. Bottom: Spike raster of recorded CST which serves as input to the model. Middle: Spike raster of the recurrent hidden layer activity. Top: Membrane potentials of the readout units (colored) and the ground-truth finger velocities (gray). D) Schematic of the training curriculum. For each monkey, we pre-trained on all available sessions, with subsequent session-wise fine-tuning. For tinyRSNN, we further added iterative pruning at the end of the curriculum.
• Figure 2: Learning and decoding performance of the RSNN models.A) Learning curves for the tinyRSNN (top) and the bigRSNN (bottom) for session I3. B)R² values for ours and the baseline models (SNN, SNN_Flatyik_neurobench_2024). Each point corresponds to a network with a different random initialization; colors indicate the different sessions.