EchoSpike Predictive Plasticity: An Online Local Learning Rule for Spiking Neural Networks

TL;DR

The paper addresses efficient online adaptation of spiking neural networks on edge devices using self-supervised, local learning. It introduces EchoSpike Predictive Plasticity (ESPP), an online local learning rule that uses an echo-based prediction and adaptive thresholds to drive layer-wise targets. ESPP achieves competitive or superior performance to existing local learning rules on N-MNIST and SHD, while enabling sparse, event-driven weight updates suitable for neuromorphic hardware. The results demonstrate scalable deep SNNs with self-supervised learning, reducing training cost and enabling edge deployment with unlabeled data.

Abstract

The drive to develop artificial neural networks that efficiently utilize resources has generated significant interest in bio-inspired Spiking Neural Networks (SNNs). These networks are particularly attractive due to their potential in applications requiring low power and memory. This potential is further enhanced by the ability to perform online local learning, enabling them to adapt to dynamic environments. This requires the model to be adaptive in a self-supervised manner. While self-supervised learning has seen great success in many deep learning domains, its application for online local learning in multi-layer SNNs remains underexplored. In this paper, we introduce the "EchoSpike Predictive Plasticity" (ESPP) learning rule, a pioneering online local learning rule designed to leverage hierarchical temporal dynamics in SNNs through predictive and contrastive coding. We validate the effectiveness of this approach using benchmark datasets, demonstrating that it performs on par with current state-of-the-art supervised learning rules. The temporal and spatial locality of ESPP makes it particularly well-suited for low-cost neuromorphic processors, representing a significant advancement in developing biologically plausible self-supervised learning models for neuromorphic computing at the edge.

Figures (5)

• Figure 1: EchoSpike Predictive Plasticity (ESPP) Compared to Related Work
• Figure 2: a) An illustration of the adaptive threshold mechanism. The background is filled with the audio data of a spoken digit (“Seven”) from the SHD dataset heidelberg. The three lines indicate the value of the adaptive threshold $\Tilde{c}(y)$, the similarity score ($\boldsymbol{s}^{t,l} \cdot \boldsymbol{\Bar{s}}^l_{prev}$) and the input threshold $i_{thr}$. The input threshold has been scaled by $c(y)$, so that it can be compared to the adaptive threshold. If the adaptive threshold is below the input threshold, no weight update will be performed. Time steps with a filled background indicate those time steps where a weight update will be performed. b) Similar to a), but for a saccade. Since the previous sample has been of a different label, the similarity score should be as small as possible. For illustration purposes, the negative of the similarity score is plotted. The adaptive threshold and the scaled input threshold are negative, because $c(-1)$ is negative.
• Figure 3: N-MNIST train and test accuracy for the gradient descent classifier (a) and from the few-shot learning experiment (b), trained for each layer separately.
• Figure 4: Train and test accuracies for the feedforward (a) and recurrent (b) architectures on the SHD dataset. We compare the four possible combinations between the two different types of classifiers (gradient descent and closed-form) and the two different types of connecting the hidden layers with the output layer (last-layer-only prediction and all-layers prediction).
• Figure 5: Train and test accuracy of a feed-forward neural network on SHD with data augmentation and gradient descent all-layers prediction.