Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AR-LIF: Adaptive reset leaky integrate-and-fire neuron for spiking neural networks

Zeyu Huang, Wei Meng, Quan Liu, Kun Chen, Li Ma

TL;DR

The paper tackles the challenge in spiking neural networks where reset modes and threshold settings trade information retention for activation control. It introduces AR-LIF, an adaptive-reset Leaky Integrate-and-Fire neuron that uses a memory-based reset and an input-dependent threshold to create heterogeneity in spiking dynamics, while maintaining energy efficiency. Through direct training with a TET-based loss and surrogate gradients, AR-LIF achieves state-of-the-art accuracy on static and neuromorphic datasets such as Tiny-ImageNet and CIFAR10-DVS, and demonstrates lower spike rates, translating to practical energy savings. The work provides extensive ablations and parameter-tracking analyses, validating the effectiveness of adaptive reset and threshold modulation and offering publicly available code for reproducibility.

Abstract

Spiking neural networks offer low energy consumption due to their event-driven nature. Beyond binary spike outputs, their intrinsic floating-point dynamics merit greater attention. Neuronal threshold levels and reset modes critically determine spike count and timing. Hard reset cause information loss, while soft reset apply uniform treatment to neurons. To address these issues, we design an adaptive reset neuron that establishes relationships between inputs, outputs, and reset, while integrating a simple yet effective threshold adjustment strategy. Experimental results demonstrate that our method achieves excellent performance while maintaining lower energy consumption. In particular, it attains state-of-the-art accuracy on Tiny-ImageNet and CIFAR10-DVS. Codes are available at https://github.com/2ephyrus/AR-LIF.

AR-LIF: Adaptive reset leaky integrate-and-fire neuron for spiking neural networks

TL;DR

The paper tackles the challenge in spiking neural networks where reset modes and threshold settings trade information retention for activation control. It introduces AR-LIF, an adaptive-reset Leaky Integrate-and-Fire neuron that uses a memory-based reset and an input-dependent threshold to create heterogeneity in spiking dynamics, while maintaining energy efficiency. Through direct training with a TET-based loss and surrogate gradients, AR-LIF achieves state-of-the-art accuracy on static and neuromorphic datasets such as Tiny-ImageNet and CIFAR10-DVS, and demonstrates lower spike rates, translating to practical energy savings. The work provides extensive ablations and parameter-tracking analyses, validating the effectiveness of adaptive reset and threshold modulation and offering publicly available code for reproducibility.

Abstract

Spiking neural networks offer low energy consumption due to their event-driven nature. Beyond binary spike outputs, their intrinsic floating-point dynamics merit greater attention. Neuronal threshold levels and reset modes critically determine spike count and timing. Hard reset cause information loss, while soft reset apply uniform treatment to neurons. To address these issues, we design an adaptive reset neuron that establishes relationships between inputs, outputs, and reset, while integrating a simple yet effective threshold adjustment strategy. Experimental results demonstrate that our method achieves excellent performance while maintaining lower energy consumption. In particular, it attains state-of-the-art accuracy on Tiny-ImageNet and CIFAR10-DVS. Codes are available at https://github.com/2ephyrus/AR-LIF.

Paper Structure

This paper contains 12 sections, 1 theorem, 12 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Leaky Integrate-and-Fire (LIF) Neuron
  4. Motivation of Adaptive Reset Neuron Design
  5. Adaptive Reset Leaky Integrate-and-Fire Neuron
  6. Experiments
  7. Experimental Details
  8. Comparisons with Existing Methods
  9. Ablation Study
  10. Analysis of Computational Efficiency
  11. Learnable Parameters Tracking
  12. Conclusion

Key Result

Theorem 1

For a spiking neuron using soft reset, the spike it fires at time $t$ can be expressed as an iterative formula of input and output, independent of the membrane potential.

Figures (3)

  • Figure 1: (a) The hollow arrows in the figure indicate the additional computational processes introduced by AR-LIF. The dashed lines represent the replication of the solid-line outputs of the nodes. (b) Spike responses to 3-channel gesture video inputs. (c) The membrane potential distribution of neurons under inputs with perturbed normal distributions.
  • Figure 2: Firing rates of different neurons across layers.
  • Figure 3: Tracking of learnable parameters in each layer.

Theorems & Definitions (2)

  • Theorem 1
  • Proof 2.1