Local Timescale Gates for Timescale-Robust Continual Spiking Neural Networks

TL;DR

LT-Gate tackles the stability-plasticity dilemma in continual learning for spiking neural networks by introducing a dual-timescale neuron with a locally learned gate, enabling simultaneous fast adaptation and long-term context retention. The method couples two parallel LIF compartments with a differentiable gate $\\gamma_i$ to blend fast and slow dynamics, and adds a variance-based homeostatic regularizer via $L_{var}$ to stabilize activity; training uses surrogate-gradient backpropagation and is compatible with neuromorphic hardware. Empirically, LT-Gate achieves about 51% final accuracy on temporally heterogeneous tasks, significantly reducing forgetting compared with baselines like HLOP and DSD-SNN, and demonstrates faster convergence and cross-timescale generalization. The work suggests multi-timescale processing as a fundamental principle for robust lifelong learning on energy-efficient neuromorphic hardware such as Loihi-2.

Abstract

Spiking neural networks (SNNs) promise energy-efficient artificial intelligence on neuromorphic hardware but struggle with tasks requiring both fast adaptation and long-term memory, especially in continual learning. We propose Local Timescale Gating (LT-Gate), a neuron model that combines dual time-constant dynamics with an adaptive gating mechanism. Each spiking neuron tracks information on a fast and a slow timescale in parallel, and a learned gate locally adjusts their influence. This design enables individual neurons to preserve slow contextual information while responding to fast signals, addressing the stability-plasticity dilemma. We further introduce a variance-tracking regularization that stabilizes firing activity, inspired by biological homeostasis. Empirically, LT-Gate yields significantly improved accuracy and retention in sequential learning tasks: on a challenging temporal classification benchmark it achieves about 51 percent final accuracy, compared to about 46 percent for a recent Hebbian continual-learning baseline and lower for prior SNN methods. Unlike approaches that require external replay or expensive orthogonalizations, LT-Gate operates with local updates and is fully compatible with neuromorphic hardware. In particular, it leverages features of Intel's Loihi chip (multiple synaptic traces with different decay rates) for on-chip learning. Our results demonstrate that multi-timescale gating can substantially enhance continual learning in SNNs, narrowing the gap between spiking and conventional deep networks on lifelong-learning tasks.

Figures (4)

• Figure 1: Architecture of the LT-Gate neuron model. Each neuron contains two parallel LIF compartments with different time constants ($\tau_f \ll \tau_s$). A learnable gate $\gamma_i$ combines their outputs. When the combined membrane potential exceeds threshold, both compartments are reset, preserving the dual-timescale dynamics.
• Figure 2: Continual learning under temporal shift and cross-timescale generalization.(a) Accuracy over 200 epochs with a fast$\to$slow task switch at epoch 100 (dashed line). LT-Gate consistently achieves higher peak accuracy pre-switch and retains more performance post-switch than HLOP and DSD-SNN, indicating reduced catastrophic forgetting. (b) After training only at 1000 Hz, accuracy is evaluated from $10$–$1000$ Hz (log scale). LT-Gate remains strongest at slow inputs and degrades most gracefully as frequency departs from the training regime, evidencing a robust multi-timescale bias.
• Figure 3: Example membrane potential dynamics for fast and slow compartments responding to two input spikes at $t=2$ms and $t=10$ms (marked by vertical dotted lines). The fast compartment (blue) responds quickly but decays rapidly. The slow compartment (red) integrates over longer periods. The gated output (green dashed) combines both pathways according to the learned gate value $\gamma = 0.7$. This demonstrates how LT-Gate neurons simultaneously capture transient events and maintain longer-term context.
• Figure 4: Learned specialization and hardware efficiency.(a) Learned gate statistics after training: early layers cluster near $\gamma\!\approx\!0.5$ (balanced integration), the hidden layer becomes bimodal, and the output layer polarizes toward $\gamma\!\approx\!0$ (fast) or $\gamma\!\approx\!1$ (slow), revealing a division of labor across timescales. (b) On Loihi 2, power scales roughly linearly with network size; LT-Gate's overhead versus a standard SNN remains small even at 1024 neurons, supporting hardware-compatible continual learning.