Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Time-independent Spiking Neuron via Membrane Potential Estimation for Efficient Spiking Neural Networks

Hanqi Chen, Lixing Yu, Shaojie Zhan, Penghui Yao, Jiankun Shao

TL;DR

The paper tackles the inefficiency of recurrent membrane-potential updates in spiking neural networks by introducing Membrane Potential Estimation Parallel Spiking Neurons (MPE-PSN), which decouple spiking from time steps through membrane-potential estimation. A Membrane Potential Approximation Loss (L_mem) aligns estimated and true potentials during training, enabling parallel computation while preserving the intrinsic spiking dynamics. Empirical results on neuromorphic datasets show state-of-the-art accuracy and notable computational speedups, e.g., achieving $97.92\%$ top-1 on DVSGesture at 16 steps, with strong performance on CIFAR10DVS and N-Caltech 101. The approach demonstrates that parallelized neuron design can leverage hardware parallelism without sacrificing dynamic properties, suggesting a viable path toward scalable, efficient SNN implementations.

Abstract

The computational inefficiency of spiking neural networks (SNNs) is primarily due to the sequential updates of membrane potential, which becomes more pronounced during extended encoding periods compared to artificial neural networks (ANNs). This highlights the need to parallelize SNN computations effectively to leverage available hardware parallelism. To address this, we propose Membrane Potential Estimation Parallel Spiking Neurons (MPE-PSN), a parallel computation method for spiking neurons that enhances computational efficiency by enabling parallel processing while preserving the intrinsic dynamic characteristics of SNNs. Our approach exhibits promise for enhancing computational efficiency, particularly under conditions of elevated neuron density. Empirical experiments demonstrate that our method achieves state-of-the-art (SOTA) accuracy and efficiency on neuromorphic datasets. Codes are available at~\url{https://github.com/chrazqee/MPE-PSN}. \end{abstract}

Time-independent Spiking Neuron via Membrane Potential Estimation for Efficient Spiking Neural Networks

TL;DR

The paper tackles the inefficiency of recurrent membrane-potential updates in spiking neural networks by introducing Membrane Potential Estimation Parallel Spiking Neurons (MPE-PSN), which decouple spiking from time steps through membrane-potential estimation. A Membrane Potential Approximation Loss (L_mem) aligns estimated and true potentials during training, enabling parallel computation while preserving the intrinsic spiking dynamics. Empirical results on neuromorphic datasets show state-of-the-art accuracy and notable computational speedups, e.g., achieving top-1 on DVSGesture at 16 steps, with strong performance on CIFAR10DVS and N-Caltech 101. The approach demonstrates that parallelized neuron design can leverage hardware parallelism without sacrificing dynamic properties, suggesting a viable path toward scalable, efficient SNN implementations.

Abstract

The computational inefficiency of spiking neural networks (SNNs) is primarily due to the sequential updates of membrane potential, which becomes more pronounced during extended encoding periods compared to artificial neural networks (ANNs). This highlights the need to parallelize SNN computations effectively to leverage available hardware parallelism. To address this, we propose Membrane Potential Estimation Parallel Spiking Neurons (MPE-PSN), a parallel computation method for spiking neurons that enhances computational efficiency by enabling parallel processing while preserving the intrinsic dynamic characteristics of SNNs. Our approach exhibits promise for enhancing computational efficiency, particularly under conditions of elevated neuron density. Empirical experiments demonstrate that our method achieves state-of-the-art (SOTA) accuracy and efficiency on neuromorphic datasets. Codes are available at~\url{https://github.com/chrazqee/MPE-PSN}. \end{abstract}
Paper Structure (11 sections, 11 equations, 4 figures, 2 tables)

This paper contains 11 sections, 11 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Motivation and Derivation of Neuron Design
  4. Initialization of Estimated Membrane Potential
  5. Membrane Potential Approximation Loss
  6. Expriments
  7. Experimental Details
  8. Comparisons With Existing Methods
  9. Membrane Potential Approximation Analysis
  10. Computational Efficiency Analysis
  11. Conclusion

Figures (4)

  • Figure 1: Variations in the execution time ratio between the vanilla-LIF neuron and the MPE-PSN neuron concerning time step increments and neuron quantity augmentation in both cases during the forward process, denoted as $\frac{T_{vanilla-LIF}}{T_{MPE-PSN}}$.
  • Figure 2: Schematic diagram of the proposed spiking neuron internal structure based on membrane potential estimation.
  • Figure 3: (a): The curves of $L_2~norm$ between $\hat{\mathbf{u}}$ and $\mathbf{u}$, as well as the accuracy curves on the DVSGesture dataset. The values are normalised to a range of 0 to 1. (b): Spike rate of MPE-PSN neurons and vanilla LIF neurons on the DVSGesture dataset.
  • Figure 4: 2D loss contours, as introduced in visualloss, along with accuracy and loss curves, are presented for the CIFAR10DVS dataset with a time step of 4.