Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Lightweight CNNs to SpikeNets: Benchmarking Accuracy-Energy Tradeoffs with Pruned Spiking SqueezeNet

Radib Bin Kabir, Tawsif Tashwar Dipto, Mehedi Ahamed, Sabbir Ahmed, Md Hasanul Kabir

TL;DR

This paper tackles the problem of deploying energy-efficient visual recognition on edge devices by benchmarking lightweight CNNs converted to Spiking Neural Networks. It introduces a unified CNN-to-SNN pipeline using Leaky-Integrate-and-Fire neurons and surrogate-gradient training, and demonstrates a pruning strategy that yields the pruned SNN-SqueezeNet-P architecture. Across CIFAR-10, CIFAR-100, and TinyImageNet, SNNs achieve substantial energy savings (up to 15.7x) with competitive accuracy, and SNN-SqueezeNet-P further narrows the gap to CNNs while reducing energy by about 5.6x. The findings provide actionable guidance for designing compact, low-power SNNs suitable for edge inference and highlight pruning as a key lever to improve accuracy and efficiency on neuromorphic hardware.

Abstract

Spiking Neural Networks (SNNs) are increasingly studied as energy-efficient alternatives to Convolutional Neural Networks (CNNs), particularly for edge intelligence. However, prior work has largely emphasized large-scale models, leaving the design and evaluation of lightweight CNN-to-SNN pipelines underexplored. In this paper, we present the first systematic benchmark of lightweight SNNs obtained by converting compact CNN architectures into spiking networks, where activations are modeled with Leaky-Integrate-and-Fire (LIF) neurons and trained using surrogate gradient descent under a unified setup. We construct spiking variants of ShuffleNet, SqueezeNet, MnasNet, and MixNet, and evaluate them on CIFAR-10, CIFAR-100, and TinyImageNet, measuring accuracy, F1-score, parameter count, computational complexity, and energy consumption. Our results show that SNNs can achieve up to 15.7x higher energy efficiency than their CNN counterparts while retaining competitive accuracy. Among these, the SNN variant of SqueezeNet consistently outperforms other lightweight SNNs. To further optimize this model, we apply a structured pruning strategy that removes entire redundant modules, yielding a pruned architecture, SNN-SqueezeNet-P. This pruned model improves CIFAR-10 accuracy by 6% and reduces parameters by 19% compared to the original SNN-SqueezeNet. Crucially, it narrows the gap with CNN-SqueezeNet, achieving nearly the same accuracy (only 1% lower) but with an 88.1% reduction in energy consumption due to sparse spike-driven computations. Together, these findings establish lightweight SNNs as practical, low-power alternatives for edge deployment, highlighting a viable path toward deploying high-performance, low-power intelligence on the edge.

From Lightweight CNNs to SpikeNets: Benchmarking Accuracy-Energy Tradeoffs with Pruned Spiking SqueezeNet

TL;DR

This paper tackles the problem of deploying energy-efficient visual recognition on edge devices by benchmarking lightweight CNNs converted to Spiking Neural Networks. It introduces a unified CNN-to-SNN pipeline using Leaky-Integrate-and-Fire neurons and surrogate-gradient training, and demonstrates a pruning strategy that yields the pruned SNN-SqueezeNet-P architecture. Across CIFAR-10, CIFAR-100, and TinyImageNet, SNNs achieve substantial energy savings (up to 15.7x) with competitive accuracy, and SNN-SqueezeNet-P further narrows the gap to CNNs while reducing energy by about 5.6x. The findings provide actionable guidance for designing compact, low-power SNNs suitable for edge inference and highlight pruning as a key lever to improve accuracy and efficiency on neuromorphic hardware.

Abstract

Spiking Neural Networks (SNNs) are increasingly studied as energy-efficient alternatives to Convolutional Neural Networks (CNNs), particularly for edge intelligence. However, prior work has largely emphasized large-scale models, leaving the design and evaluation of lightweight CNN-to-SNN pipelines underexplored. In this paper, we present the first systematic benchmark of lightweight SNNs obtained by converting compact CNN architectures into spiking networks, where activations are modeled with Leaky-Integrate-and-Fire (LIF) neurons and trained using surrogate gradient descent under a unified setup. We construct spiking variants of ShuffleNet, SqueezeNet, MnasNet, and MixNet, and evaluate them on CIFAR-10, CIFAR-100, and TinyImageNet, measuring accuracy, F1-score, parameter count, computational complexity, and energy consumption. Our results show that SNNs can achieve up to 15.7x higher energy efficiency than their CNN counterparts while retaining competitive accuracy. Among these, the SNN variant of SqueezeNet consistently outperforms other lightweight SNNs. To further optimize this model, we apply a structured pruning strategy that removes entire redundant modules, yielding a pruned architecture, SNN-SqueezeNet-P. This pruned model improves CIFAR-10 accuracy by 6% and reduces parameters by 19% compared to the original SNN-SqueezeNet. Crucially, it narrows the gap with CNN-SqueezeNet, achieving nearly the same accuracy (only 1% lower) but with an 88.1% reduction in energy consumption due to sparse spike-driven computations. Together, these findings establish lightweight SNNs as practical, low-power alternatives for edge deployment, highlighting a viable path toward deploying high-performance, low-power intelligence on the edge.
Paper Structure (22 sections, 4 figures, 3 tables)

This paper contains 22 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. ANN-to-SNN Conversion Methods
  4. Lightweight CNN Architectures
  5. Pruning Techniques in Neural Networks
  6. Evaluation of SNNs on Standard Benchmarks
  7. Gaps and Contributions
  8. Methodology
  9. Datasets
  10. SNN Conversion
  11. Model Architecture
  12. Experimental Setup and Training Strategy
  13. Evaluation Metrics
  14. Result Analysis
  15. Performance Comparison Among SNN Models
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Accuracy-energy trade-off on CIFAR-10. While CNNs achieve strong accuracy, they incur much higher energy costs. SNNs reduce energy but often sacrifice accuracy. Our pruned SNN SqueezeNet-P (orange star) lies on the Pareto frontier, delivering the best balance between accuracy and efficiency.
  • Figure 2: Comparison of neuron-level processing in ANN and SNN (reproduced from Acharya2020).
  • Figure 3: Pruning strategy for SNN SqueezeNet. Each 'Fi' denotes a Fire (firing) module 'i'. The top row shows the original SNN SqueezeNet with all Fire modules active. The bottom row illustrates the pruned SNN SqueezeNet, where ineffective Fire modules (F2, F3, F5, F7) are removed, and only selected modules (F4, F6, F8, F9) are kept. The dotted green line indicates that the retained Fire modules maintain their functional connections, ensuring information flow through the pruned network.
  • Figure 4: Gradient flow visualization of SNN-SqueezeNet before and after structured pruning. The pruning process removes modules that accumulate vanishing gradients and restores effective gradient propagation.