Sup3r: A Semi-Supervised Algorithm for increasing Sparsity, Stability, and Separability in Hierarchy Of Time-Surfaces architectures

TL;DR

This work tackles the need for end-to-end, online training of Hierarchy Of Time-Surfaces (HOTS) without external classifiers by introducing Sup3r, a semi-supervised learning rule that enhances sparsity, stability, and separability. Sup3r employs a local online learning update guided by a class-discriminative descriptor S and a per-layer feedback mechanism, enabling continual and incremental learning while reducing processed events. The approach is validated on a synthetic 3-layer benchmark where Sup3r achieves near-ANN accuracy with far fewer events, and demonstrates continual and incremental learning capabilities; preliminary NMNIST results show competitive performance with backpropagation on a small network. Overall, Sup3r demonstrates the potential to train HOTS end-to-end in neuromorphic settings, improving efficiency and adaptability for real-world event-based tasks, with ongoing work to accelerate GPU implementations and scale to deeper architectures.

Abstract

The Hierarchy Of Time-Surfaces (HOTS) algorithm, a neuromorphic approach for feature extraction from event data, presents promising capabilities but faces challenges in accuracy and compatibility with neuromorphic hardware. In this paper, we introduce Sup3r, a Semi-Supervised algorithm aimed at addressing these challenges. Sup3r enhances sparsity, stability, and separability in the HOTS networks. It enables end-to-end online training of HOTS networks replacing external classifiers, by leveraging semi-supervised learning. Sup3r learns class-informative patterns, mitigates confounding features, and reduces the number of processed events. Moreover, Sup3r facilitates continual and incremental learning, allowing adaptation to data distribution shifts and learning new tasks without forgetting. Preliminary results on N-MNIST demonstrate that Sup3r achieves comparable accuracy to similarly sized Artificial Neural Networks trained with back-propagation. This work showcases the potential of Sup3r to advance the capabilities of HOTS networks, offering a promising avenue for neuromorphic algorithms in real-world applications.

Figures (3)

• Figure 1: The visual classification benchmark to test Sup3r on a small 3-layered network. a) The test is composed of a "sentence" (blue) classification task, where two different sentences are each composed of two among three different "words" (green), which are composed of three of five different "characters" (orange). The only difference between the two sentences consists of two characters, "x" and "/", which are contained in the top word. To generate spiking data, sentences are written by a lattice of 5x30 Poisson neurons firing at a high firing rate for the yellow pixels and a low firing rate for the blue pixels. b) The convolutional HOTS network used to solve the task. The gray rectangles represent the output dimensionality of each layer. $N$ is the number of clusters responding to the six different characters in a sentence (Layer 1), to the two words in a sentence (Layer 2), and finally to the entire sentence (Layer 3).
• Figure 2: Neuromorphic benchmark results, comparing Sup3r against $k$-means for feature extraction. a) Layer 1 features extracted with $k$-means (light blue) and Sup3r (orange). $k$-means fails to extract class-relevant features, while Sup3r can correctly extract "x" "/", leaving one centroid unused. b) Consequently, $k$-means fails the classification task, performing close to the chance level ($\approx$55.36% accuracy, calculated as the percentage of events assigned to the correct class), while Sup3r can solve the task with $\approx$99.92% accuracy. Since Sup3r rejects events from non-class-relevant characters, only $\approx$14.33% of events are propagated in the network (calculated as the ratio between the number of the last layer output events divided by the number of the first layer input events). Conversely, the k-means network outputs the same number of input events.
• Figure 3: Continual learning test with Sup3r. In this test, we perform a class shift in the two sentences by subtracting pixels from "x" and adding them to "/" every 250 sentences (a), with the final shift lasting 1000 sentences. Sup3r centroids slowly adapt to the shift (b). In (c), Sup3r network accuracy remains high and returns to >99%, while the same network without unsupervised learning fails the task (Ablation). This test is performed with ten independently trained networks and randomized test sets. Accuracy is reported as the average across runs in dark colors with min and max values in light colors.