Understanding the Functional Roles of Modelling Components in Spiking Neural Networks

TL;DR

This work systematically investigates the functional roles of three core LIF-based SNN components—leakage, reset, and recurrence—through ablation variants and extensive benchmarks. By discretizing the LIF dynamics and evaluating across temporal, spatial, and spatio-temporal datasets, the authors quantify how leakage balances memory retention and robustness, how reset supports uninterrupted temporal processing and efficiency, and how recurrence enhances complex temporal dynamics at the cost of robustness and generalization. Across accuracy, generalization, and adversarial robustness, the study finds leakage to be a key driver of performance, recurrence to introduce richer but less robust dynamics, and reset to offer efficiency benefits with nuanced interactions. The results yield actionable optimization guidance for tailoring SNN designs to specific tasks and hardware, contributing to more effective and robust neuromorphic models.

Abstract

Spiking neural networks (SNNs), inspired by the neural circuits of the brain, are promising in achieving high computational efficiency with biological fidelity. Nevertheless, it is quite difficult to optimize SNNs because the functional roles of their modelling components remain unclear. By designing and evaluating several variants of the classic model, we systematically investigate the functional roles of key modelling components, leakage, reset, and recurrence, in leaky integrate-and-fire (LIF) based SNNs. Through extensive experiments, we demonstrate how these components influence the accuracy, generalization, and robustness of SNNs. Specifically, we find that the leakage plays a crucial role in balancing memory retention and robustness, the reset mechanism is essential for uninterrupted temporal processing and computational efficiency, and the recurrence enriches the capability to model complex dynamics at a cost of robustness degradation. With these interesting observations, we provide optimization suggestions for enhancing the performance of SNNs in different scenarios. This work deepens the understanding of how SNNs work, which offers valuable guidance for the development of more effective and robust neuromorphic models.

Figures (8)

• Figure 1: Illustration of the LIF-based spiking neuron model: (a) spike integration, transformation, and generation in a spiking neuron; (b) network structure of SNNs highlighting temporal and spatial dimensions; (c) leakage component reflecting membrane potential decaying; (d) reset component reflecting the membrane potential reset after each spike; (e) recurrence reflecting cross-neuron influence in the temporal dimension.
• Figure 2: Variant models of LIF-based SNNs: (a) vanilla LIF; (b) without leakage; (c) complete leakage; (d) without reset; (e) with recurrence.
• Figure 3: Average spike rate distribution of variant models on N-MNIST: (a) normal reset; (b) without reset.
• Figure 4: Spatio-temporal data collected by DVS with variable temporal integration lengths.
• Figure 5: Comparison of the loss landscapes: (a) vanilla SNN; (b) variant SNN without leakage and with recurrence; (c) LSTM.