Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spiking Layer-Adaptive Magnitude-based Pruning

Junqiao Wang, Zhehang Ye, Yuqi Ouyang

Abstract

Spiking Neural Networks (SNNs) provide energy-efficient computation but their deployment is constrained by dense connectivity and high spiking operation costs. Existing magnitude-based pruning strategies, when naively applied to SNNs, fail to account for temporal accumulation, non-uniform timestep contributions, and membrane stability, often leading to severe performance degradation. This paper proposes Spiking Layer-Adaptive Magnitude-based Pruning (SLAMP), a theory-guided pruning framework that generalizes layer-adaptive magnitude pruning to temporal SNNs by explicitly controlling worst-case output distortion across layers and timesteps. SLAMP formulates sparsity allocation as a temporal distortion-constrained optimization problem, yielding time-aware layer importance scores that reduce to conventional layer-adaptive pruning in single-timestep limit. An efficient two-stage procedure is derived, combining temporal score estimation, global sparsity allocation, and magnitude pruning with retraining for stability recovery. Experiments on CIFAR10, CIFAR100, and the event-based CIFAR10-DVS datasets demonstrate that SLAMP achieves substantial connectivity and spiking operation reductions while preserving accuracy, enabling efficient and deployable SNN inference.

Spiking Layer-Adaptive Magnitude-based Pruning

Abstract

Spiking Neural Networks (SNNs) provide energy-efficient computation but their deployment is constrained by dense connectivity and high spiking operation costs. Existing magnitude-based pruning strategies, when naively applied to SNNs, fail to account for temporal accumulation, non-uniform timestep contributions, and membrane stability, often leading to severe performance degradation. This paper proposes Spiking Layer-Adaptive Magnitude-based Pruning (SLAMP), a theory-guided pruning framework that generalizes layer-adaptive magnitude pruning to temporal SNNs by explicitly controlling worst-case output distortion across layers and timesteps. SLAMP formulates sparsity allocation as a temporal distortion-constrained optimization problem, yielding time-aware layer importance scores that reduce to conventional layer-adaptive pruning in single-timestep limit. An efficient two-stage procedure is derived, combining temporal score estimation, global sparsity allocation, and magnitude pruning with retraining for stability recovery. Experiments on CIFAR10, CIFAR100, and the event-based CIFAR10-DVS datasets demonstrate that SLAMP achieves substantial connectivity and spiking operation reductions while preserving accuracy, enabling efficient and deployable SNN inference.
Paper Structure (11 sections, 8 equations, 2 figures, 3 tables, 1 algorithm)

This paper contains 11 sections, 8 equations, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Spiking Neural Networks
  4. Temporal Distortion–Aware Sparsity Allocation
  5. Experiments
  6. Datasets
  7. Implementation Details
  8. Performance Comparison
  9. Ablation Study of Fine-Tuning
  10. Hyperparameter Analysis
  11. Conclusion

Figures (2)

  • Figure 1: Structures of the two SNNs studied in this work.
  • Figure 2: A complete illustration of the pruning performance across the three datasets, including pruned model density (histograms) and resulted Top-1 accuracy (red curve) and Top-5 accuracy (green curve).