Efficient Synaptic Delay Implementation in Digital Event-Driven AI Accelerators

TL;DR

This work tackles the memory and energy challenges of implementing synaptic delays in digital event-driven neuromorphic accelerators. It introduces the Shared Circular Delay Queue (SCDQ), a two-FIFO, circular delay structure with a pruning filter, and demonstrates its hardware realization as Delay IP on the Seneca platform. The results show strong fidelity to software models, substantial reductions in energy and latency compared to baseline delay implementations and Loihi, and significant area reductions through SRAM-backed memory. These findings indicate that synaptic delay parameterizations can be memory-efficient and practically advantageous for edge AI, especially when combined with hardware-algorithm co-optimizations that leverage activation sparsity and pruning.

Abstract

Synaptic delay parameterization of neural network models have remained largely unexplored but recent literature has been showing promising results, suggesting the delay parameterized models are simpler, smaller, sparser, and thus more energy efficient than similar performing (e.g. task accuracy) non-delay parameterized ones. We introduce Shared Circular Delay Queue (SCDQ), a novel hardware structure for supporting synaptic delays on digital neuromorphic accelerators. Our analysis and hardware results show that it scales better in terms of memory, than current commonly used approaches, and is more amortizable to algorithm-hardware co-optimizations, where in fact, memory scaling is modulated by model sparsity and not merely network size. Next to memory we also report performance on latency area and energy per inference.

Figures (8)

• Figure 1: Example of two SNN delay models with two presynaptic neurons, two postynaptic neurons, and three levels of delay.
• Figure 2: An example of the event flow over three timesteps in a two-layer SNN with synaptic delays. The maximum delay is 2, and the Shared Circular Delay Queue is positioned between the two layers. In timestep $t=0$, neurons $A$ and $B$ spike, and neuron $C$ receives spikes from neurons $A$ and $B$ with a delay value of 0. In timestep $t=1$, neuron $B$ spikes, and neuron $C$ receives a spike from neuron $A$ with a delay value of 0, from neuron $B$ with a delay value of 0, and from neuron $B$ with a delay value of 1. In timestep $t=2$, no neuron spikes, and neuron $C$ receives a spike from neuron $A$ with a delay value of 2, from neuron $B$ with a delay value of 2, and from neuron $B$ with a delay value of 1.
• Figure 3: Example of $WVU$ for a network where delayed axons $A,2$, $B,0$ and $B,1$ are skipped.
• Figure 4: A mapping on the Seneca platform of a three-layer SNN with 700 wide input samples, 48 neurons in both hidden layers, 20 neurons in the output layer, and 60 delays between the hidden layers
• Figure 5: Activations of a datapoint in all layers for 48-48-20 network on Seneca, Loihi and PyTorch