A feedback control optimizer for online and hardware-aware training of Spiking Neural Networks

TL;DR

A novel learning algorithm for Spiking Neural Networks (SNNs) on mixed-signal devices that integrates spike-based weight updates with feedback control signals is presented, advancing the potential for scalable, on-chip learning solutions in edge applications.

Abstract

Unlike traditional artificial neural networks (ANNs), biological neuronal networks solve complex cognitive tasks with sparse neuronal activity, recurrent connections, and local learning rules. These mechanisms serve as design principles in Neuromorphic computing, which addresses the critical challenge of energy consumption in modern computing. However, most mixed-signal neuromorphic devices rely on semi- or unsupervised learning rules, which are ineffective for optimizing hardware in supervised learning tasks. This lack of scalable solutions for on-chip learning restricts the potential of mixed-signal devices to enable sustainable, intelligent edge systems. To address these challenges, we present a novel learning algorithm for Spiking Neural Networks (SNNs) on mixed-signal devices that integrates spike-based weight updates with feedback control signals. In our framework, a spiking controller generates feedback signals to guide SNN activity and drive weight updates, enabling scalable and local on-chip learning. We first evaluate the algorithm on various classification tasks, demonstrating that single-layer SNNs trained with feedback control achieve performance comparable to artificial neural networks (ANNs). We then assess its implementation on mixed-signal neuromorphic devices by testing network performance in continuous online learning scenarios and evaluating resilience to hyperparameter mismatches. Our results show that the feedback control optimizer is compatible with neuromorphic applications, advancing the potential for scalable, on-chip learning solutions in edge applications.

Figures (4)

• Figure 1: Illustration of the feedback control optimizer and binary classification task. a) Illustration of the feedback control architecture. A neuron receives external input spikes (black and grey arrows) associated with learnable synaptic weights ($W_A$ and $W_B$) and, in turn, sends output spikes to the control module (blue arrows). The neuron is paired with a positive (magenta) and a negative (light blue) control neuron. The positive and negative control neurons receive inhibitory and excitatory activity from the output neuron, respectively. The control neurons also receive external target spikes (purple arrows) and, in turn, send feedback spikes (light purple arrows). b) Illustration of the binary classification task. Bottom) The activity of the pre-synaptic inputs A and B for class 1 (pink) and 2 (orange), respectively. Top-left) The input-output function (f-f curve) of the neuron. The neuron has two target activities corresponding to the two classes (target 1 and target 2) and is trained to match these targets when inputs from the respective classes are presented. Top-right) Evolution of the learnable synaptic weights $W_A$ and $W_B$ during training. c) Illustration of the feedback control algorithm. Example spiking activities of the neuron, the targets, and the positive and negative control for an example input from class 1 (top) and class 2 (bottom). d) Left: the average cross-entropy loss during training (black) and validation (purple). Right: the mean absolute difference between output and target activity during validation. e) Average firing rate during validation for inputs from class 1 (black) and class 2 (light red). Left: neuron encoding class 1. Right: neuron encoding class 2. We calculate the mean and standard deviation across multiple simulations with different random seeds.
• Figure 2: Training a single-layer SNN on the spiking Yin-Yang dataset with feedback control. a) Illustration of the spiking Yin-Yang dataset and the corresponding targets. Each dot belongs to the Yin (purple), Yang (azure), or dot (pink) region. Here we illustrate the spiking patterns of the input coordinates (bottom) and target activities (top) of an example dot from the Yang class. b) The average validation accuracy for the feedback control network (purple) compared to a network of three leaky-integrator neurons trained with backpropagation-through-time (BPTT). The dashed blue line represents test accuracy for a standard readout layer trained with backpropagation, using firing rates directly as features for each dot's coordinates. c) Left) The network predictions on the test set, with a test accuracy of 0.63 $\pm$ 0.03. Right) The predictions of a standard ANN on the test set, with a test accuracy of 0.66 $\pm$ 0.02. d) The mean firing rate of the output neurons at the final epoch for both training (train) and validation (val) inputs across the Yin, Yang, and Dot classes. Each output neuron is color-coded according to the class it represents: purple for Yin, azure for Yang, and pink for Dot. e) The mean target error on the test set, with dots color-coded by associated target error (see color bar). We calculate the mean and standard deviation across multiple simulations with different random seeds.
• Figure 3: Online learning in a single-layer SNN with feedback control. a) Schematic of the online learning setup. (b-d) Binary classification tasks. b) The average cross-entropy loss during training (black) and validation (purple). We calculate the mean and standard deviation by averaging over 25 sequential inputs. c) Same as b for the mean absolute difference between output and target activity during validation. d) Same as b for firing rate during validation for inputs from class A (black) and class B (grey) for a neuron encoding class A. (e-h) The Spiking Yin-Yang dataset: e) The average cross-entropy loss during training (black) and validation (purple). We calculate the mean and standard deviation by averaging over 50 sequential inputs. f) Same as e for the mean absolute difference between output and target activity during validation. The test target error is calculated offline over the whole testing set with batch size 20. g) Same as e for the average classification accuracy during validation. h) The average network predictions when testing offline over the whole testing set with batch size 20. We calculate the mean and standard deviation across multiple simulations with different random seeds.
• Figure 4: Robustness of the feedback control optimizer to device mismatch. a) The test classification accuracy for different values of simulated device mismatch (in percentage). The performance are calculated with and without population averaging with $p$ = 2 (purple and pink, respectively). The median and inter-quartile ranges are calculated over 15 simulations with different random seeds. b) Same as a for the test target error. c) Same as a for the validation loss. d) The test classification accuracy for different population sizes with a given mismatch (20%).