NSPDI-SNN: An efficient lightweight SNN based on nonlinear synaptic pruning and dendritic integration

TL;DR

NSPDI-SNN targets the efficiency gap in spiking neural networks by combining nonlinear dendritic integration (NDI) with nonlinear synaptic pruning (NSP). Formally, NDI enriches input integration with a nonlinear term, e.g., $I = W x + b + (W x) \odot (V x)$, for both dense and convolutional layers; NSP reparameterizes weights via $w_p = \operatorname{sign}(\theta)\,[ a(|\theta| - d_1) ]_+$ and introduces a pruning threshold $d_2$ to drive sparsity. Through spike-timing backpropagation with a surrogate gradient and a mean-squared error objective, NSPDI achieves controllable sparsity while preserving accuracy, leveraging channel-level transition gains to balance performance and efficiency. Across datasets and tasks including DVS128 Gesture, CIFAR10-DVS, CIFAR10, SHD, and Maze2D, NSPDI-SNN demonstrates high sparsity with minimal performance degradation and notable energy and parameter reductions, indicating practical applicability to edge and neuromorphic hardware.

Abstract

Spiking neural networks (SNNs) are artificial neural networks based on simulated biological neurons and have attracted much attention in recent artificial intelligence technology studies. The dendrites in biological neurons have efficient information processing ability and computational power; however, the neurons of SNNs rarely match the complex structure of the dendrites. Inspired by the nonlinear structure and highly sparse properties of neuronal dendrites, in this study, we propose an efficient, lightweight SNN method with nonlinear pruning and dendritic integration (NSPDI-SNN). In this method, we introduce nonlinear dendritic integration (NDI) to improve the representation of the spatiotemporal information of neurons. We implement heterogeneous state transition ratios of dendritic spines and construct a new and flexible nonlinear synaptic pruning (NSP) method to achieve the high sparsity of SNN. We conducted systematic experiments on three benchmark datasets (DVS128 Gesture, CIFAR10-DVS, and CIFAR10) and extended the evaluation to two complex tasks (speech recognition and reinforcement learning-based maze navigation task). Across all tasks, NSPDI-SNN consistently achieved high sparsity with minimal performance degradation. In particular, our method achieved the best experimental results on all three event stream datasets. Further analysis showed that NSPDI significantly improved the efficiency of synaptic information transfer as sparsity increased. In conclusion, our results indicate that the complex structure and nonlinear computation of neuronal dendrites provide a promising approach for developing efficient SNN methods.

Figures (9)

• Figure 1: Schematic illustration of nonlinear state transitions in dendritic spines and dendritic integration in biological neurons. (a) State transitions between mature dendritic spines and filopodia, as well as between excitatory and inhibitory synapses in biological neurons. Inhibitory synapses (left, blue) utilize $\gamma$-aminobutyric acid (GABA) as their primary neurotransmitter, whereas excitatory synapses (right, green) rely on glutamate. The nonlinear weight reparameterization, inspired by these synaptic transitions, is depicted in the function plot in the upper left. (b) Dendritic integration of synaptic inputs is inherently nonlinear. The resulting membrane potential is modeled as the linear summation of input potentials augmented by a nonlinear correction term.
• Figure 2: NDI and NSP. (a) An NDI-LIF model. When input spiking trains ($\mathbf{s}_1$, $\mathbf{s}_2$, $\mathbf{s}_3$) are fed into neurons, the current is integrated with the correction term $\Delta I$ (yellow). Then, through the soma (green), an output spiking train ($\mathbf{s}$) is issued. (b) Complete scheme of NSP. First, an initial SNN model consisting of NDI-LIF neurons. The model's weights are reparametrized to complete the initial pruning. Then, a threshold pruning strategy is used to reduce the weighted connections further.
• Figure 3: Average top-1 performance of different SNN models. Comparison of the baseline SNN (B, green) and three NDI-SNN variants—synapse-level (S, orange), channel-level (C, light orange), and layer-level (L, yellow)—on three datasets: (a) DVS128 Gesture, (b) CIFAR10-DVS, and (c) CIFAR10. Blue dots indicate the best top-1 accuracy achieved across independent trials.
• Figure 4: Accuracy difference with respect to the baseline sparse model (Linear + STDS) under varying sparsity levels on three datasets. Each line corresponds to one model: NSPDI (red), Linear + NSP(yellow), and NDI + STDS (green). (a) DVS128 Gesture dataset. (b) CIFAR10-DVS dataset. (c) CIFAR10 dataset.
• Figure 5: Accuracy comparison of NSPDI-SNN models using transition gains introduced at different structural levels: synapse-level (S, blue), channel-level(C, teal), and layer-level (L, green) levels. The red dashed line represents the accuracy of NDI-SNN using the STDS method. Each plot corresponds to one dataset: (a) DVS128 Gesture. (b) CIFAR10-DVS. (c) CIFAR10.