General Self-Prediction Enhancement for Spiking Neurons

TL;DR

This work tackles the training challenges of Spiking Neural Networks (SNNs) by introducing self-prediction enhanced spiking neurons that generate an internal prediction current from a neuron’s input-output history. The prediction current, updated via a low-pass filter, modulates the membrane potential to pre-activate expected spikes or dampen noise-driven firing, thereby creating a continuous gradient path and improving training stability. The authors analyze gradient flow, provide a biological interpretation tied to distal dendritic modulation and local prediction errors, and validate the method across image classification, sequential tasks, and reinforcement learning. The results show broad performance gains across architectures and neuron types, underscoring the approach’s generality and potential impact for efficient and biologically plausible SNN training.

Abstract

Spiking Neural Networks (SNNs) are highly energy-efficient due to event-driven, sparse computation, but their training is challenged by spike non-differentiability and trade-offs among performance, efficiency, and biological plausibility. Crucially, mainstream SNNs ignore predictive coding, a core cortical mechanism where the brain predicts inputs and encodes errors for efficient perception. Inspired by this, we propose a self-prediction enhanced spiking neuron method that generates an internal prediction current from its input-output history to modulate membrane potential. This design offers dual advantages, it creates a continuous gradient path that alleviates vanishing gradients and boosts training stability and accuracy, while also aligning with biological principles, which resembles distal dendritic modulation and error-driven synaptic plasticity. Experiments show consistent performance gains across diverse architectures, neuron types, time steps, and tasks demonstrating broad applicability for enhancing SNNs.

Figures (12)

• Figure 1: Left: original neuron, Right: our proposed self-prediction enhanced neuron. Neurons operate in the sequence of charging, firing, and resetting at each time-step.
• Figure 2: Forward propagation pathway of the self-prediction enhanced LIF neuron. The red lines indicate the additional forward path associated with self-prediction.
• Figure 3: Backward propagation pathway of the self-prediction enhanced LIF neuron. The blue and green lines form the first additional gradient path, and the blue and red lines form the second additional gradient path.The dashed lines indicate paths that are detached from the computational graph.
• Figure 4: The training and testing accuracy curves of four spiking neuron models, IF, LIF, PLIF, and CLIF, along with their variants enhanced by self-prediction mechanisms.
• Figure 5: Normalized learning curves across all environments of the TD3 algorithm with different spiking neurons across all environments. The performance and training steps are normalized linearly based on ANN performance. Curves are uniformly smoothed for visual clarity.