Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks

TL;DR

This work tackles two core challenges in directly trained deep SNNs: gradient vanishing from non-differentiable spikes and accuracy degradation in very deep networks. It introduces multi-level firing (MLF) to broaden effective gradient regions and enrich neuron expression, and spiking dormant-suppressed residual networks (spiking DS-ResNet) to preserve identity mappings while suppressing dormant units. Together, MLF and DS-ResNet enable very deep, efficiently parameterized SNNs that achieve state-of-the-art results on CIFAR10 and neuromorphic datasets (DVS-Gesture, CIFAR10-DVS) with far fewer parameters. The approach yields improved gradient flow, allows deeper architectures without degradation, and demonstrates strong potential for energy-efficient neuromorphic applications. Overall, the paper provides a practical, scalable path to high-performance directly trained SNNs by combining advanced gradient signaling with refined residual connectivity.

Abstract

Spiking neural networks (SNNs) are bio-inspired neural networks with asynchronous discrete and sparse characteristics, which have increasingly manifested their superiority in low energy consumption. Recent research is devoted to utilizing spatio-temporal information to directly train SNNs by backpropagation. However, the binary and non-differentiable properties of spike activities force directly trained SNNs to suffer from serious gradient vanishing and network degradation, which greatly limits the performance of directly trained SNNs and prevents them from going deeper. In this paper, we propose a multi-level firing (MLF) method based on the existing spatio-temporal back propagation (STBP) method, and spiking dormant-suppressed residual network (spiking DS-ResNet). MLF enables more efficient gradient propagation and the incremental expression ability of the neurons. Spiking DS-ResNet can efficiently perform identity mapping of discrete spikes, as well as provide a more suitable connection for gradient propagation in deep SNNs. With the proposed method, our model achieves superior performances on a non-neuromorphic dataset and two neuromorphic datasets with much fewer trainable parameters and demonstrates the great ability to combat the gradient vanishing and degradation problem in deep SNNs.

Figures (9)

• Figure 1: Illustration of MLF unit. A MLF unit contains multiple LIF neurons with different level thresholds. After receiving the input, these neurons will update the membrane potentials. Once the membrane potential of each level neuron reaches the corresponding threshold, a spike will be fired. The final output of MLF unit is the union of the spikes fired by all level neurons.
• Figure 2: (a) The distribution of approximate derivative of one-level firing. (b) The distribution of approximate derivatives of multi-level firing.
• Figure 3: Illustration of spiking ResNet and spiking DS-ResNet. The dotted line connection represents spiking ResNet, and the solid line connection represents spiking DS-ResNet.
• Figure 4: (a) The average proportion of dormant units in each layer. (b) The training loss during the whole training process.
• Figure 5: Training on CIFAR10. Thin dotted curves denote testing accuracy, and bold solid curves denote training accuracy. (a) ResNet-SNN without MLF. (b) ResNet-SNN with MLF. (c) Spiking DS-ResNet with MLF.

Theorems & Definitions (4)

• Theorem 1
• proof
• Theorem 2
• proof