Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LightSNN: Lightweight Architecture Search for Sparse and Accurate Spiking Neural Networks

Yesmine Abdennadher, Giovanni Perin, Riccardo Mazzieri, Jacopo Pegoraro, Michele Rossi

TL;DR

The paper addresses the challenge of suboptimal accuracy when applying SNNs with ANN-like architectures by introducing LightSNN, a rapid NAS framework tailored for sparse, energy-efficient spiking networks. It builds on SNASNet with a pruning-by-importance strategy over a supernet and sparsity-aware SAHD evaluation to dramatically cut search time while boosting accuracy. To promote sparsity and efficiency, LightSNN refines the search space by replacing average pooling with max pooling and limiting operations to 3x3 convolutions, skip connections, and zeroize. Empirical results on CIFAR-10, CIFAR-100, and DVS128 Gesture show substantial accuracy gains and up to 98x speedup over SNASNet, with strong performance on neuromorphic data and favorable energy-related metrics. Overall, LightSNN offers a practical, energy-conscious NAS approach for high-performing, sparse SNNs suitable for edge devices.

Abstract

Spiking Neural Networks (SNNs) are highly regarded for their energy efficiency, inherent activation sparsity, and suitability for real-time processing in edge devices. However, most current SNN methods adopt architectures resembling traditional artificial neural networks (ANNs), leading to suboptimal performance when applied to SNNs. While SNNs excel in energy efficiency, they have been associated with lower accuracy levels than traditional ANNs when utilizing conventional architectures. In response, in this work we present LightSNN, a rapid and efficient Neural Network Architecture Search (NAS) technique specifically tailored for SNNs that autonomously leverages the most suitable architecture, striking a good balance between accuracy and efficiency by enforcing sparsity. Based on the spiking NAS network (SNASNet) framework, a cell-based search space including backward connections is utilized to build our training-free pruning-based NAS mechanism. Our technique assesses diverse spike activation patterns across different data samples using a sparsity-aware Hamming distance fitness evaluation. Thorough experiments are conducted on both static (CIFAR10 and CIFAR100) and neuromorphic datasets (DVS128-Gesture). Our LightSNN model achieves state-of-the-art results on CIFAR10 and CIFAR100, improves performance on DVS128Gesture by 4.49\%, and significantly reduces search time most notably offering a $98\times$ speedup over SNASNet and running 30\% faster than the best existing method on DVS128Gesture. Code is available on Github at: https://github.com/YesmineAbdennadher/LightSNN.

LightSNN: Lightweight Architecture Search for Sparse and Accurate Spiking Neural Networks

TL;DR

The paper addresses the challenge of suboptimal accuracy when applying SNNs with ANN-like architectures by introducing LightSNN, a rapid NAS framework tailored for sparse, energy-efficient spiking networks. It builds on SNASNet with a pruning-by-importance strategy over a supernet and sparsity-aware SAHD evaluation to dramatically cut search time while boosting accuracy. To promote sparsity and efficiency, LightSNN refines the search space by replacing average pooling with max pooling and limiting operations to 3x3 convolutions, skip connections, and zeroize. Empirical results on CIFAR-10, CIFAR-100, and DVS128 Gesture show substantial accuracy gains and up to 98x speedup over SNASNet, with strong performance on neuromorphic data and favorable energy-related metrics. Overall, LightSNN offers a practical, energy-conscious NAS approach for high-performing, sparse SNNs suitable for edge devices.

Abstract

Spiking Neural Networks (SNNs) are highly regarded for their energy efficiency, inherent activation sparsity, and suitability for real-time processing in edge devices. However, most current SNN methods adopt architectures resembling traditional artificial neural networks (ANNs), leading to suboptimal performance when applied to SNNs. While SNNs excel in energy efficiency, they have been associated with lower accuracy levels than traditional ANNs when utilizing conventional architectures. In response, in this work we present LightSNN, a rapid and efficient Neural Network Architecture Search (NAS) technique specifically tailored for SNNs that autonomously leverages the most suitable architecture, striking a good balance between accuracy and efficiency by enforcing sparsity. Based on the spiking NAS network (SNASNet) framework, a cell-based search space including backward connections is utilized to build our training-free pruning-based NAS mechanism. Our technique assesses diverse spike activation patterns across different data samples using a sparsity-aware Hamming distance fitness evaluation. Thorough experiments are conducted on both static (CIFAR10 and CIFAR100) and neuromorphic datasets (DVS128-Gesture). Our LightSNN model achieves state-of-the-art results on CIFAR10 and CIFAR100, improves performance on DVS128Gesture by 4.49\%, and significantly reduces search time most notably offering a speedup over SNASNet and running 30\% faster than the best existing method on DVS128Gesture. Code is available on Github at: https://github.com/YesmineAbdennadher/LightSNN.

Paper Structure

This paper contains 14 sections, 2 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Baseline approaches
  2. Spiking neuron dynamics
  3. SNASNet
  4. Cell-based search space
  5. Training-free NAS approach
  6. LightSNN: rationale and methods
  7. Pruning-by-importance algorithm
  8. Lightweight-and-sparsity-aware NAS
  9. Search space refinements
  10. Experiments and evaluation
  11. Datasets
  12. Hyperparameters
  13. Results
  14. Conclusion

Figures (2)

  • Figure 1: Example of a candidate cell.
  • Figure 2: Pruning by importance steps. Pruned operations are shown with a light-gray color.