LoAS: Fully Temporal-Parallel Dataflow for Dual-Sparse Spiking Neural Networks

TL;DR

A fully temporal-parallel (FTP) dataflow that minimizes data movement across timesteps and reduces the end-to-end latency of dual-sparse SNNs is proposed and an FTP-friendly spike compression mechanism that efficiently compresses single-bit spikes and ensures contiguous memory access is introduced.

Abstract

Spiking Neural Networks (SNNs) have gained significant research attention in the last decade due to their potential to drive resource-constrained edge devices. Though existing SNN accelerators offer high efficiency in processing sparse spikes with dense weights, opportunities are less explored in SNNs with sparse weights, i.e., dual-sparsity. In this work, we study the acceleration of dual-sparse SNNs, focusing on their core operation, sparse-matrix-sparse-matrix multiplication (spMspM). We observe that naively running a dual-sparse SNN on existing spMspM accelerators designed for dual-sparse Artificial Neural Networks (ANNs) exhibits sub-optimal efficiency. The main challenge is that processing timesteps, a natural property of SNNs, introduces an extra loop to ANN spMspM, leading to longer latency and more memory traffic. To address the problem, we propose a fully temporal-parallel (FTP) dataflow, which minimizes both data movement across timesteps and the end-to-end latency of dual-sparse SNNs. To maximize the efficiency of FTP dataflow, we propose an FTP-friendly spike compression mechanism that efficiently compresses single-bit spikes and ensures contiguous memory access. We further propose an FTP-friendly inner-join circuit that can lower the cost of the expensive prefix-sum circuits with almost no throughput penalty. All the above techniques for FTP dataflow are encapsulated in LoAS, a Low-latency inference Accelerator for dual-sparse SNNs. With FTP dataflow, compression, and inner-join, running dual-sparse SNN workloads on LoAS demonstrates significant speedup (up to $8.51\times$) and energy reduction (up to $3.68\times$) compared to running it on prior dual-sparse accelerators.

Figures (19)

• Figure 1: An illustrative example of FTP dataflow and LoAS. FTP dataflow is shown along with the prior dataflow design for SNNs. Temporal sequential tick-batch is from SpinalFlow spinalflow, and partially temporal parallel is from PTB ptb. Each arrow loop indicates the processing of one timestep. The vertical line indicates that the processing is in parallel.
• Figure 2: Difference between the LIF-based SNN neuron and ReLU-based ANN neuron. We compare the behavior of LIF-based and ReLU-based neurons and their hardware implementations.
• Figure 3: Comparison of different spMspM dataflow for SNNs. Here, for illustration purposes, we put $C[t_i]$ as the spMspM result between $A[t_i]$ and $B$ to align with spMspM in ANNs. In SNNs, we need to go through one more LIF step (Equation (2)) to get $C[t_i]$. The circled numbers illustrate the order of computation for the specific spMspM dataflow. Please note that we fix the position of t dimension for illustration purposes. In practice, there will be a total of 16 possible permutations of spMspM dataflow in SNNs, which we will discuss in Section \ref{['sec:dataflow']}.
• Figure 4: An example of the inner-join design. The difference between the behavior of ANN and SNN is shown. *Data from SparTen sparten.
• Figure 5: Off-chip traffic of partial sum matrices on different SNN layers. We envision SNNs with a timestep of 1 and 4 running on GoSPA gospa, an OP dataflow spMspM accelerator.