Exploiting Heterogeneity in Timescales for Sparse Recurrent Spiking Neural Networks for Energy-Efficient Edge Computing

TL;DR

This work tackles energy-efficient edge computing with Spiking Neural Networks by introducing heterogeneity in both neuron and STDP dynamics (HRSNN) and a Lyapunov-based pruning method (LNP). It provides analytical backing showing that neuronal heterogeneity enhances memory capacity while STDP heterogeneity reduces spike activity, leading to improved spike efficiency. A modified Bayesian Optimization approach tunes distributions over hyperparameters, and LNP yields sparse, robust networks with minimal performance loss across tasks. Collectively, the findings demonstrate substantial potential for practical neuromorphic implementations that achieve high performance with lower energy costs and improved robustness.

Abstract

Spiking Neural Networks (SNNs) represent the forefront of neuromorphic computing, promising energy-efficient and biologically plausible models for complex tasks. This paper weaves together three groundbreaking studies that revolutionize SNN performance through the introduction of heterogeneity in neuron and synapse dynamics. We explore the transformative impact of Heterogeneous Recurrent Spiking Neural Networks (HRSNNs), supported by rigorous analytical frameworks and novel pruning methods like Lyapunov Noise Pruning (LNP). Our findings reveal how heterogeneity not only enhances classification performance but also reduces spiking activity, leading to more efficient and robust networks. By bridging theoretical insights with practical applications, this comprehensive summary highlights the potential of SNNs to outperform traditional neural networks while maintaining lower computational costs. Join us on a journey through the cutting-edge advancements that pave the way for the future of intelligent, energy-efficient neural computing.

Figures (12)

• Figure 1: Concept of HRSNN with variable Neuronal and Synaptic Dynamics
• Figure 2: An illustrative example showing the Heterogeneous Recurrent Spiking Neural Network structure. First, we show the temporal encoding method based on the sensory receptors receiving the difference between two time-adjacent data. Next, the input sequences are encoded by the encoding neurons that inject the spike train into $30\%$ neurons in $\mathcal{R}. \mathcal{R}$ contains a $4:1$ ratio of excitatory (green nodes) and inhibitory (orange nodes), where the neuron parameters are heterogeneous. The synapses are trained using the heterogeneous STDP method.
• Figure 3: (a) Concept of HRSNN with variable Neuronal Dynamics (b) Figure showing the task-agnostic pruning and training of the CHRSNN/HRSNN networks using LNP in comparison to the current approach
• Figure 4: Bar chart showing the global importance of different heterogeneous parameters using HRSNN on the dataset. The experiments were repeated five times with different parameters from the same distribution (a) Classification (b) Prediction
• Figure 5: Comparison of performance of HRSNN models for the (a) KTH dataset and (b) DVS128 dataset for varying number of neurons. The bar graph (left Y-axis) shows the difference between the accuracies between HeNHeS and HoNHoS models. The line graphs (right Y-axis) shows the Accuracies (%) for the four ablation networks (HoNHoS, HeNHoS, HoNHeS, HeNHeS)