Paired Competing Neurons Improving STDP Supervised Local Learning In Spiking Neural Networks

TL;DR

This work tackles energy-efficient, on-chip training of Spiking Neural Networks by combining unsupervised STDP feature learning with a locally trained supervised classifier. It introduces Stabilized Supervised STDP (S2-STDP), which uses dynamically computed desired timestamps based on the layer mean to guide one-spike-per-neuron updates, and Paired Competing Neurons (PCN) to promote class-specific neuron specialization. empirically, S2-STDP outperforms SSTDP across MNIST, Fashion-MNIST, and CIFAR-10, with PCN providing additional gains without adding hyperparameters and showing robustness to hyperparameter variations. The results suggest that end-to-end local learning on neuromorphic hardware is feasible with competitive accuracy and improved training stability, opening pathways for more scalable on-chip SNN training.

Abstract

Direct training of Spiking Neural Networks (SNNs) on neuromorphic hardware has the potential to significantly reduce the energy consumption of artificial neural network training. SNNs trained with Spike Timing-Dependent Plasticity (STDP) benefit from gradient-free and unsupervised local learning, which can be easily implemented on ultra-low-power neuromorphic hardware. However, classification tasks cannot be performed solely with unsupervised STDP. In this paper, we propose Stabilized Supervised STDP (S2-STDP), a supervised STDP learning rule to train the classification layer of an SNN equipped with unsupervised STDP for feature extraction. S2-STDP integrates error-modulated weight updates that align neuron spikes with desired timestamps derived from the average firing time within the layer. Then, we introduce a training architecture called Paired Competing Neurons (PCN) to further enhance the learning capabilities of our classification layer trained with S2-STDP. PCN associates each class with paired neurons and encourages neuron specialization toward target or non-target samples through intra-class competition. We evaluate our methods on image recognition datasets, including MNIST, Fashion-MNIST, and CIFAR-10. Results show that our methods outperform state-of-the-art supervised STDP learning rules, for comparable architectures and numbers of neurons. Further analysis demonstrates that the use of PCN enhances the performance of S2-STDP, regardless of the hyperparameter set and without introducing any additional hyperparameters.

Figures (9)

• Figure 1: Architecture of the SNN employed in this paper for image recognition tasks. First, the image is preprocessed and each pixel is encoded into a single floating-point timestamp using the latency coding scheme. Then, a Convolutional SNN (CSNN) trained with unsupervised STDP is used to extract relevant features from the image. The resulting feature maps are compressed through a max-pooling layer to reduce their size and provide invariance to translation on the input. Lastly, they are flattened and fed to a fully-connected SNN trained with a supervised adaptation of STDP for classification. Each output neuron is associated with a class and the first one to fire predicts the label. Training is done in a layer-wise fashion. This classification pipeline organized into 3 blocks provides a flexible framework for SNNs combining feature extraction and classification. In this paper, we focus on the classification layer block (C), which may be integrated after other encoding or feature extraction blocks based on latency coding.
• Figure 2: Update ratio and train accuracy per epoch in the classification layer trained with SSTDP. Training with SSTDP results in a limited number of STDP updates per epoch, which may lead to premature training convergence and suboptimal model performance.
• Figure 3: Average firing time per epoch in the classification layer trained with SSTDP. Training with SSTDP causes the saturation of firing timestamps toward the maximum firing time, which may limit the expressivity of the SNN and its ability to separate classes.
• Figure 4: Classification layer equipped with Paired Competing Neurons (PCN) and trained via Stabilized Supervised STDP (S2-STDP). Each class is represented by paired neurons, interconnected with lateral inhibition to create intra-class competition. Within a pair, the first neuron to fire (the winner) inhibits the other one (the loser) and undergoes the STDP update. The difference $\Delta t$ between the desired and actual firing timestamps is used to compute the neuron error, which modulates the intensity and the polarity of the STDP update. The purpose of PCN is to enhance the learning capabilities of the neurons by promoting specialization toward target or non-target samples.
• Figure 5: Update ratio, average firing time, and train accuracy per epoch in the classification layer trained on MNIST. Our methods using S2-STDP significantly increase the number of updates per epoch and reduce the saturation of firing timestamps toward the maximum firing time. As a result, they enable training convergence at higher accuracies compared to SSTDP.