STAER: Temporal Aligned Rehearsal for Continual Spiking Neural Network

TL;DR

STAER addresses class-incremental learning for Spiking Neural Networks by preserving spike-timing information through a differentiable Soft-DTW alignment loss and a temporal expansion/contraction replay mechanism on a deep Spiking ResNet19 backbone. The method integrates an experience replay buffer with the temporal alignment objective to maintain temporal fidelity of outputs across tasks, mitigating catastrophic forgetting. Experiments on Sequential-MNIST and Sequential-CIFAR10 show STAER achieving state-of-the-art results among spiking CL methods and competitive performance with ANN baselines, particularly at larger time horizons $T$. Ablation studies confirm that explicit temporal alignment is crucial for representational stability, positioning STAER as a scalable spike-native solution for lifelong learning and a path toward neuromorphic continual learning on more complex tasks.

Abstract

Spiking Neural Networks (SNNs) are inherently suited for continuous learning due to their event-driven temporal dynamics; however, their application to Class-Incremental Learning (CIL) has been hindered by catastrophic forgetting and the temporal misalignment of spike patterns. In this work, we introduce Spiking Temporal Alignment with Experience Replay (STAER), a novel framework that explicitly preserves temporal structure to bridge the performance gap between SNNs and ANNs. Our approach integrates a differentiable Soft-DTW alignment loss to maintain spike timing fidelity and employs a temporal expansion and contraction mechanism on output logits to enforce robust representation learning. Implemented on a deep ResNet19 spiking backbone, STAER achieves state-of-the-art performance on Sequential-MNIST and Sequential-CIFAR10. Empirical results demonstrate that our method matches or outperforms strong ANN baselines (ER, DER++) while preserving biologically plausible dynamics. Ablation studies further confirm that explicit temporal alignment is critical for representational stability, positioning STAER as a scalable solution for spike-native lifelong learning. Code is available at https://github.com/matteogianferrari/staer.

Figures (3)

• Figure 1: For each input, logits are stored at three temporal resolutions $(T, T/2, 2T)$ in the replay buffer to mimic biological memory variability. The training objective combines cross-entropy on current task samples with a Temporal Alignment (TA) loss based on Soft-DTW, computed between current and past logits at multiple temporal scales.
• Figure 2: Spiking ResNet19 backbone.Left: A static input image is replicated over $T$ time steps to form a $[T, B, C, H, W]$ tensor, then processed by an initial convolutional stem followed by a sequence of spiking residual blocks. The network ends with global average pooling and an MLP head, producing real-valued logits at every time step ($[T, B, 10]$ for our datasets). Right: Detail of a spiking ResNet block: the main branch stacks convolution--normalization layers interleaved with LIF neurons, while a residual shortcut is added to the main branch output before the next spiking activation.
• Figure 3: Hyperparameter sensitivity of the Temporal Alignment (TA) objective on Sequential-MNIST. Each heatmap (fixed $\beta$) reports the final CIL accuracy (lighter color is better) as a function of the SDTW compression/dilation weights $(\alpha_1,\alpha_2)$ and the TA strength $\beta$. The x-axis shows $\alpha_1$ (compression), the y-axis shows $\alpha_2$ (dilation).