Neuronal Competition Groups with Supervised STDP for Spike-Based Classification

TL;DR

This paper proposes a method to effectively implement WTA competition in a spiking classification layer employing first-spike coding and supervised STDP training, and introduces the Neuronal Competition Group (NCG), an architecture that improves classification capabilities by promoting the learning of various patterns per class.

Abstract

Spike Timing-Dependent Plasticity (STDP) is a promising substitute to backpropagation for local training of Spiking Neural Networks (SNNs) on neuromorphic hardware. STDP allows SNNs to address classification tasks by combining unsupervised STDP for feature extraction and supervised STDP for classification. Unsupervised STDP is usually employed with Winner-Takes-All (WTA) competition to learn distinct patterns. However, WTA for supervised STDP classification faces unbalanced competition challenges. In this paper, we propose a method to effectively implement WTA competition in a spiking classification layer employing first-spike coding and supervised STDP training. We introduce the Neuronal Competition Group (NCG), an architecture that improves classification capabilities by promoting the learning of various patterns per class. An NCG is a group of neurons mapped to a specific class, implementing intra-class WTA and a novel competition regulation mechanism based on two-compartment thresholds. We incorporate our proposed architecture into spiking classification layers trained with state-of-the-art supervised STDP rules. On top of two different unsupervised feature extractors, we obtain significant accuracy improvements on image recognition datasets such as CIFAR-10 and CIFAR-100. We show that our competition regulation mechanism is crucial for ensuring balanced competition and improved class separation.

Figures (3)

• Figure 1: Spiking classification layer with Neuronal Competition Groups (NCGs). In this layer, each class is mapped to an NCG and the prediction is based on the first spike. An NCG is a group of $M$ neurons connected with lateral inhibition to enable intra-class WTA competition: the first neuron to fire inhibits the other ones and undergoes a weight update based on a temporal error (which depends on the learning rule considered). The sign and amplitude of the error pushes neurons to fire earlier (positive sign) or later (negative sign). Competition regulation occurs only within the NCG mapped to the class of the input sample to ensure balanced competition among neurons on samples of their class. NCGs improve the classification capabilities of a layer by promoting the learning of various patterns per class.
• Figure 2: Number of weight updates per epoch received by the neurons of class $0$ trained with S2-STDP+NCG, with and without competition regulation, on CIFAR-10. $n_1$ to $n_4$ are labeled as target neurons and $n_5$ is labeled as non-target. The features are extracted with STDP-CSNN.
• Figure 3: t-SNE plots of the weights learned with S2-STDP+NCG on CIFAR-10, with and without competition regulation. Crosses and circles respectively represent the weights of target and non-target neurons, and colors indicate classes. The features are extracted with STDP-CSNN.