Multiplication-Free Parallelizable Spiking Neurons with Efficient Spatio-Temporal Dynamics

TL;DR

The paper tackles the bottleneck of multiplication-heavy, serial spiking neuron dynamics that hinder GPU-friendly training of SNNs. It introduces the mul-free channel-wise PSN, which uses channel-wise and dilated convolutions with sawtooth dilations to enlarge temporal receptive fields while keeping neuron order small, and replaces multiplications with hardware-friendly bit-shifts by quantizing weights to powers of two. It also develops training-acceleration strategies, including an autoselect method to pick the fastest data-layout and acceleration approach, and demonstrates state-of-the-art performance on temporal tasks such as SHD, sequential CIFAR, and DVS-Lip, with reduced inference memory and energy. The work offers a practical, hardware-aware solution to improve both the speed and efficiency of training and deploying SNNs on neuromorphic hardware and GPUs for long-range temporal processing.

Abstract

Spiking Neural Networks (SNNs) are distinguished from Artificial Neural Networks (ANNs) for their complex neuronal dynamics and sparse binary activations (spikes) inspired by the biological neural system. Traditional neuron models use iterative step-by-step dynamics, resulting in serial computation and slow training speed of SNNs. Recently, parallelizable spiking neuron models have been proposed to fully utilize the massive parallel computing ability of graphics processing units to accelerate the training of SNNs. However, existing parallelizable spiking neuron models involve dense floating operations and can only achieve high long-term dependencies learning ability with a large order at the cost of huge computational and memory costs. To solve the dilemma of performance and costs, we propose the mul-free channel-wise Parallel Spiking Neuron, which is hardware-friendly and suitable for SNNs' resource-restricted application scenarios. The proposed neuron imports the channel-wise convolution to enhance the learning ability, induces the sawtooth dilations to reduce the neuron order, and employs the bit-shift operation to avoid multiplications. The algorithm for the design and implementation of acceleration methods is discussed extensively. Our methods are validated in neuromorphic Spiking Heidelberg Digits voices, sequential CIFAR images, and neuromorphic DVS-Lip vision datasets, achieving superior performance over SOTA spiking neurons. Training speed results demonstrate the effectiveness of our acceleration methods, providing a practical reference for future research. Our code is available at \href{https://github.com/PengXue0812/Multiplication-Free-Parallelizable-Spiking-Neurons-with-Efficient-Spatio-Temporal-Dynamics}{Github}.

Figures (7)

• Figure 1: The neuronal dynamics of the mul-free channel-wise Parallel Spiking Neuron.
• Figure 2: The temporal receptive field increases with depths at a slow rate in the sliding PSN with (a) fixed dilations and a fast rate in the channel-wise PSN with (b) sawtooth dilations.
• Figure 3: Power-of-2 quantization and the gradient behavior of $\mathrm{round}'(x)$ under STE.
• Figure 4: Reshape operations involving adjacent dimensions are free of memory reading/writing and are much faster than those involving nonadjacent dimensions.
• Figure 5: The order-accuracy curves on (a) the sequential CIFAR100 and (b) the pixel CIFAR10.