Parallel Spiking Unit for Efficient Training of Spiking Neural Networks

TL;DR

This work tackles the fundamental bottleneck of sequential processing in Spiking Neural Networks (SNNs) by introducing Parallel Spiking Unit (PSU) and its variants IPSU and RPSU, which decouple leak-integration from firing and approximate the reset probabilistically to enable parallel computation of membrane potentials $V[t]$ across all time steps. The PSU family employs causal masking and spike-estimation $S' = AI$ to achieve high sparsity and faster simulation, while IPSU and RPSU further tailor the leak and reset dynamics through learned matrices $A_I$ and $A_R$. Extensive experiments across static and sequential images (CIFAR-10/100), DVS data (DVSCIFAR10), and speech (SHD) show that PSU variants deliver improved speed, reduced firing rates, and competitive or superior accuracy compared with traditional LIF and prior parallel SNN approaches. These results underscore the potential for efficient, energy-conscious SNN deployment on parallel hardware and neuromorphic platforms. The combination of parallelizable membrane dynamics, data-adaptive leakage/reset, and strong empirical validation suggests a practical pathway toward scalable, low-power SNNs for real-time AI tasks.

Abstract

Efficient parallel computing has become a pivotal element in advancing artificial intelligence. Yet, the deployment of Spiking Neural Networks (SNNs) in this domain is hampered by their inherent sequential computational dependency. This constraint arises from the need for each time step's processing to rely on the preceding step's outcomes, significantly impeding the adaptability of SNN models to massively parallel computing environments. Addressing this challenge, our paper introduces the innovative Parallel Spiking Unit (PSU) and its two derivatives, the Input-aware PSU (IPSU) and Reset-aware PSU (RPSU). These variants skillfully decouple the leaky integration and firing mechanisms in spiking neurons while probabilistically managing the reset process. By preserving the fundamental computational attributes of the spiking neuron model, our approach enables the concurrent computation of all membrane potential instances within the SNN, facilitating parallel spike output generation and substantially enhancing computational efficiency. Comprehensive testing across various datasets, including static and sequential images, Dynamic Vision Sensor (DVS) data, and speech datasets, demonstrates that the PSU and its variants not only significantly boost performance and simulation speed but also augment the energy efficiency of SNNs through enhanced sparsity in neural activity. These advancements underscore the potential of our method in revolutionizing SNN deployment for high-performance parallel computing applications.

Figures (6)

• Figure 1: The computational process of the vanilla LIF neuron model and the parallel spiking units. Due to the Reset process, LIF neurons cannot be computed in parallel. In contrast, parallel spiking units decouple the Reset process from the Leak-Integrate process by estimating the required spike outputs, allowing for the parallel computation of outputs at all times. In the diagram, $H$ represents the Heaviside function, and $\sigma$ denotes the Sigmoid function.
• Figure 2: Comparison of PSU, IPSU, and RPSU. They both use causal masking techniques to mask the effect of information from future time steps on the current moment. RPSU learns parametrically the matrix $A_R$ involved in the reset part of the process from the point of view of better spike estimation. In contrast, IPSU learns parametrically the matrix $A_I$ responsible for the leakage integration from the point of view of better leakage integration. Both intend to preserve the reset process's features during parallelization and better adapt to the data through parametric learning.
• Figure 3: Example of Datasets. (a) CIFAR10 (b) Sequential CIFAR10 (c) SHD (d) DVSCIFAR10
• Figure 4: The Simulation Time Ratio of the Training and Testing Phase on SHD and DVSCIFAR10 datasets.
• Figure 5: Analysis of neural activity in VGGSNN on DVSCIFAR10, including (a) firing rates and (b) feature maps.