Real-time Continual Learning on Intel Loihi 2

TL;DR

This work addresses online continual learning (OCL) on edge devices under open-world conditions, where distributions shift and novel classes appear. It proposes CLP-SNN, a spiking neural network on Intel Loihi 2 that realizes continual learning through event-driven local updates, a self-normalizing three-factor rule $\Delta w = \alpha r (\mathbf{x} - \mathbf{w} y)$ with $y = \mathbf{w}^T \mathbf{x}$, neurogenesis for capacity expansion, and metaplasticity to mitigate forgetting. On OpenLORIS, CLP-SNN matches or surpasses replay-based baselines while being rehearsal-free, and achieves 70× lower learning latency and 5,600× higher energy efficiency than the best edge-GPU OCL method, with additional gains from input sparsity and temporal sparsity. The work includes cross-platform benchmarking, a principled derivation of the learning rule, and open-source simulation code, illustrating the potential of co-designing brain-inspired algorithms with neuromorphic hardware for real-time edge AI.

Abstract

AI systems on edge devices face a critical challenge in open-world environments: adapting when data distributions shift and novel classes emerge. While offline training dominates current paradigms, online continual learning (OCL)--where models learn incrementally from non-stationary streams without catastrophic forgetting--remains challenging in power-constrained settings. We present a neuromorphic solution called CLP-SNN: a spiking neural network architecture for Continually Learning Prototypes and its implementation on Intel's Loihi 2 chip. Our approach introduces three innovations: (1) event-driven and spatiotemporally sparse local learning, (2) a self-normalizing three-factor learning rule maintaining weight normalization, and (3) integrated neurogenesis and metaplasticity for capacity expansion and forgetting mitigation. On OpenLORIS few-shot learning experiments, CLP-SNN achieves accuracy competitive with replay methods while being rehearsal-free. CLP-SNN delivers transformative efficiency gains: 70\times faster (0.33ms vs 23.2ms), and 5,600\times more energy efficient (0.05mJ vs 281mJ) than the best alternative OCL on edge GPU. This demonstrates that co-designed brain-inspired algorithms and neuromorphic hardware can break traditional accuracy-efficiency trade-offs for future edge AI systems.

Figures (3)

• Figure 1: From dense global updates to event‑driven local learning on neuromorphic chip Loihi 2, as an efficient solution to online continual learning.a Catastrophic forgetting happens when learning tasks (e.g. objects) are presented sequentially in non-i.i.d data streams. Online Continual Learning (OCL) is such a setting, where inference and learning occur per sample. b (Left) Global learning with backpropagation relies on non‑local error signals (orange dashed arrows) and updates all weights every step (orange flashes), shown only for two example weights (blue arrows). This produces spatially and temporally dense learning, which is inefficient. (Middle) Local Hebbian learning relies solely on local information, making it more efficient; however, it is enacted at each timestep for all neurons, remaining spatially and temporally dense. (Right) Event-driven local learning triggers weight updates only when and where spikes occur (e.g., at $t^1_{spike}$ for the first weight and $t^2_{spike}$ for the second weight), resulting in spatially and temporally sparse updates that facilitate efficient and rapid learning. To achieve such learning, CLP-SNN employs a 3-factor local learning rule triggered by a selective modulatory signal. c CLP-SNN proposes a neurogenesis mechanism to increase capacity on demand as new concepts are learned, and d metaplasticity, which modulates plasticity over time (from very plastic to consolidated) to address catastrophic forgetting. e Intel Loihi 2 neuromorphic chip with neurocores supporting on-chip, event-driven communication across axons, neurons, and synapses, enables local, sparse learning.
• Figure 2: The proposed spiking neural network (SNN) architecture for the CLP algorithm. a The SNN consists of multiple neural populations, where the prototype neurons are central. Each prototype neuron has its plasticity level that controls the scale of the update applied to its input synapses. The prototypes compete using a winner-take-all mechanism implemented through lateral inhibition. For each sample, only one (winner) prototype neuron updates its weights. If no neuron spikes, the input sample is detected as novel by a novelty detector neuron, which triggers a modulator neuron, which in turn triggers learning in one of the unallocated prototype neurons. If there was a winning neuron, its label is considered the inferred label, and the supervisor sends a feedback signal based on the correctness of this inference. This is relayed to the prototype population via the modulator neuron to update the respective prototype neuron. b A pictographic view of the feature space and representation of the prototypes in this space. Each class may have a variable number of prototype neurons. c Temporal dynamics of inference, novelty detection, feedback, and learning in SNN time step resolution. The first sample of "laptop" triggers the allocation of a new (transparent) purple prototype neuron, while the second sample updates this neuron's synapses further. The dynamics are depicted for one of this neuron's input synapses over the time for processing of two samples.
• Figure 3: Few-shot online continual learning experiments with OpenLORIS dataset.a The comparison of the trends for average accuracy rate on observed classes during the 1-shot learning experiment. While offline learning methods (grays) fail, OCL algorithms retain accuracy, and CLP outperforms the baselines. The gradual decay of accuracy is a result of the increase in task difficulty (more classes to differentiate among). b 25-shot OCL experiment. Each time step corresponds to one shot. CLP outperforms all methods except SLDA, which, thanks to its use of the covariance matrix, can leverage the data better, albeit with higher latency and energy costs (see Table \ref{['table:1']}). In both experiments, CLP-SNN slightly underperforms CLP, as a result of the weight/activation quantization and discretized temporal WTA. c Energy efficiency versus accuracy trade-offs for online continual learning algorithms across different hardware architectures. Accuracy values are recorded at the end of 25-shot learning experiment. The dashed diagonal line represents the standard OCL performance boundary in conventional computing architectures. CLP on the GPU/CPU already improves this boundary thanks to its algorithmic innovations, such as multi-prototype learning and adaptive learning rates. This is further improved significantly ($\sim200\times$) by Loihi 2 implementation. d. The learning throughput versus accuracy trade-off yields results similar to the other (c) trade-off. e Learning throughput versus energy efficiency.