Efficient Implementation of a Multi-Layer Gradient-Free Online-Trainable Spiking Neural Network on FPGA

TL;DR

The hardware-optimised implementation of ODESA is shown to preserve the performance of the original algorithm across multiple spatial-temporal classification problems with significantly reduced hardware requirements.

Abstract

This paper presents an efficient hardware implementation of the recently proposed Optimized Deep Event-driven Spiking Neural Network Architecture (ODESA). ODESA is the first network to have end-to-end multi-layer online local supervised training without using gradients and has the combined adaptation of weights and thresholds in an efficient hierarchical structure. This research shows that the network architecture and the online training of weights and thresholds can be implemented efficiently on a large scale in hardware. The implementation consists of a multi-layer Spiking Neural Network (SNN) and individual training modules for each layer that enable online self-learning without using back-propagation. By using simple local adaptive selection thresholds, a Winner-Takes-All (WTA) constraint on each layer, and a modified weight update rule that is more amenable to hardware, the trainer module allocates neuronal resources optimally at each layer without having to pass high-precision error measurements across layers. All elements in the system, including the training module, interact using event-based binary spikes. The hardware-optimized implementation is shown to preserve the performance of the original algorithm across multiple spatial-temporal classification problems with significantly reduced hardware requirements.

Figures (19)

• Figure 1: Multi-Layer Supervision in ODESA using Spike-Timing-Dependent Threshold Adaptation. The shaded vertical lines represent the binary Global Attention Signal generated for each output label spike. The dotted vertical lines represent the binary Local Attention Signals sent to each layer from its next layer. The up and down arrows represent the reward and punishment of the individual neurons. Case 1: The predicted output spike matches the label spike, and the corresponding output neuron is rewarded. Case 2: The corresponding output neuron for the correct class is punished as it failed to spike in the presence of input from Layer 2. Case 3: All neurons in Layer 2 are punished as they failed to spike for an input spike from Layer 1 in the presence of the Global Attention Signal. Case 4: The active neuron in Layer 2 is rewarded in the presence of the Global Attention Signal. Case 5: The neurons with trace above the resent threshold are rewarded and the other neurons are punished in the presence of Local Attention Signal from Layer 2. Figure reproduced from YESH.
• Figure 2: ODESA training algorithm for a hidden layer.
• Figure 3: ODESA training algorithm for an output layer
• Figure 4: Synchronizer module and its timing diagram. The 'i_spike' signal will be synchronized with the rising edge of 'i_clk'. If 'i_rst_n' is not activated new spike will be ignored.
• Figure 5: Leaky Accumulator architecture with linear decay and the circuit timing diagram with two subsequent input spikes.