Toward Efficient Deep Spiking Neuron Networks:A Survey On Compression
Hui Xie, Ge Yang, Wenjuan Gao
TL;DR
This survey addresses the need for efficient Deep Spiking Neural Networks (DSNNs) by focusing on compression techniques that reduce energy and computation on neuromorphic hardware. It catalogs DSNN-specific methods—pruning (unstructured, structured, time-step), quantization (weight, membrane potential, binary), knowledge distillation (SNN–SNN and ANN–SNN), and firing-rate reduction (frequency and temporal coding)—and discusses how these are adapted from DANNs to accommodate spike-based temporal dynamics. The authors also review the biological and computational units underlying DSNNs, emphasizing differences from ANNs and the LIF model through equations such as $ au_m rac{dV}{dt} = V_{reset} - V + R_m I$ and related discrete-time updates. Finally, they outline future directions for efficient DSNNs, including efficient neuron models, unified compression architectures, hardware co-design, and specialized DSNN development strategies to bridge biology and engineering in neuromorphic contexts.
Abstract
With the rapid development of deep learning, Deep Spiking Neural Networks (DSNNs) have emerged as promising due to their unique spike event processing and asynchronous computation. When deployed on neuromorphic chips, DSNNs offer significant power advantages over Deep Artificial Neural Networks (DANNs) and eliminate time and energy consuming multiplications due to the binary nature of spikes (0 or 1). Additionally, DSNNs excel in processing temporal information, making them potentially superior for handling temporal data compared to DANNs. However, their deep network structure and numerous parameters result in high computational costs and energy consumption, limiting real-life deployment. To enhance DSNNs efficiency, researchers have adapted methods from DANNs, such as pruning, quantization, and knowledge distillation, and developed specific techniques like reducing spike firing and pruning time steps. While previous surveys have covered DSNNs algorithms, hardware deployment, and general overviews, focused research on DSNNs compression and efficiency has been lacking. This survey addresses this gap by concentrating on efficient DSNNs and their compression methods. It begins with an exploration of DSNNs' biological background and computational units, highlighting differences from DANNs. It then delves into various compression methods, including pruning, quantization, knowledge distillation, and reducing spike firing, and concludes with suggestions for future research directions.