Cannistraci-Hebb Training on Ultra-Sparse Spiking Neural Networks

TL;DR

The paper tackles the challenge of achieving ultra-high structural sparsity in spiking neural networks without sacrificing accuracy. It introduces CH-SNN, a four-stage sparse training framework featuring Sparse Spike Correlated Topological Initialization, Sparse Spike Weight Initialization, hybrid Link Removal Score pruning, and CH3-L3 regrowth. The approach demonstrates strong sparsity-accuracy trade-offs across six datasets and several architectures, and achieves substantial energy efficiency on neuromorphic hardware through the S-TP platform. This work provides a practical pathway toward ultra-efficient edge AI with SNNs and a topology-guided regrowth mechanism that can inspire future hardware-friendly sparse learning methods.

Abstract

Inspired by the brain's spike-based computation, spiking neural networks (SNNs) inherently possess temporal activation sparsity. However, when it comes to the sparse training of SNNs in the structural connection domain, existing methods fail to achieve ultra-sparse network structures without significant performance loss, thereby hindering progress in energy-efficient neuromorphic computing. This limitation presents a critical challenge: how to achieve high levels of structural connection sparsity while maintaining performance comparable to fully connected networks. To address this challenge, we propose the Cannistraci-Hebb Spiking Neural Network (CH-SNN), a novel and generalizable dynamic sparse training framework for SNNs consisting of four stages. First, we propose a sparse spike correlated topological initialization (SSCTI) method to initialize a sparse network based on node correlations. Second, temporal activation sparsity and structural connection sparsity are integrated via a proposed sparse spike weight initialization (SSWI) method. Third, a hybrid link removal score (LRS) is applied to prune redundant weights and inactive neurons, improving information flow. Finally, the CH3-L3 network automaton framework inspired by Cannistraci-Hebb learning theory is incorporated to perform link prediction for potential synaptic regrowth. These mechanisms enable CH-SNN to achieve sparsification across all linear layers. We have conducted extensive experiments on six datasets including CIFAR-10 and CIFAR-100, evaluating various network architectures such as spiking convolutional neural networks and Spikformer.

Figures (4)

• Figure 1: The framework of Cannistraci-Hebb Spiking Neural Network (CH-SNN).
• Figure 2: Performance comparison of different methods on MNIST, N-MNIST and CIFAR-10. We plot the performance of different sparse SNN training methods with structural sparsity on the x-axis and accuracy improvement on the y-axis. The plot clearly shows that CH-SNN achieves the highest level of sparsity alongside the greatest improvement in accuracy.
• Figure 3: Experimental results of CH-SNN on hardware-friendly algorithm S-TP. A comparison of firing rates between the sparse network with CH-SNN and the FC network across four datasets is presented in the left plot. The middle plot provides an accuracy comparison among the FC network and sparse networks at sparsity levels of 80%, 95%, and 99%. The right plot displays the energy consumption comparison, with the vertical axis on a base-100 logarithmic scale. A scale difference of 0.99 signifies that the energy consumption of the FC network is 97.5 ($100^{0.99}$) times greater than that of the sparse network.
• Figure 4: Example of link prediction using CH3-L3.