Emulating Brain-like Rapid Learning in Neuromorphic Edge Computing

TL;DR

This work tackles the challenge of personalized, real-time learning at the edge under non-i.i.d. streaming data. It introduces a two-stage framework where offline bi-level optimization meta-trains a local three-factor plasticity rule (SOEL) for digital neuromorphic hardware, and on-device deployment uses these hyperparameters to enable fast, data-efficient one-shot learning with on-chip updates. The approach is demonstrated on event-based vision tasks using the Intel Loihi processor, showing that real-time one-shot adaptation can be achieved with sub-1 mW dynamic power and tens of milliseconds per update, outperforming transfer-learning baselines. The results suggest a scalable path toward robust edge learning with brain-inspired plasticity, while acknowledging limitations such as catastrophic forgetting and outlining future extensions to federated and continual learning on larger neuromorphic platforms.

Abstract

Achieving personalized intelligence at the edge with real-time learning capabilities holds enormous promise in enhancing our daily experiences and helping decision making, planning, and sensing. However, efficient and reliable edge learning remains difficult with current technology due to the lack of personalized data, insufficient hardware capabilities, and inherent challenges posed by online learning. Over time and across multiple developmental stages, the brain has evolved to efficiently incorporate new knowledge by gradually building on previous knowledge. In this work, we emulate the multiple stages of learning with digital neuromorphic technology that simulates the neural and synaptic processes of the brain using two stages of learning. First, a meta-training stage trains the hyperparameters of synaptic plasticity for one-shot learning using a differentiable simulation of the neuromorphic hardware. This meta-training process refines a hardware local three-factor synaptic plasticity rule and its associated hyperparameters to align with the trained task domain. In a subsequent deployment stage, these optimized hyperparameters enable fast, data-efficient, and accurate learning of new classes. We demonstrate our approach using event-driven vision sensor data and the Intel Loihi neuromorphic processor with its plasticity dynamics, achieving real-time one-shot learning of new classes that is vastly improved over transfer learning. Our methodology can be deployed with arbitrary plasticity models and can be applied to situations demanding quick learning and adaptation at the edge, such as navigating unfamiliar environments or learning unexpected categories of data through user engagement.

Figures (4)

• Figure 1: Our approach consists of two stages of learning whereby an offline stage uses off-the-self deep learning accelerators and GPUs to optimize the hyperparameters $\phi$ of a simulation of the learning hardware, and a deployment stage where the online weights $w$ are tuned to solve a new task. This bi-level learning setup can be viewed as a method to inject the inductive biases in a given task domain (e.g. gestures recorded with event-based vision sensors) necessary to facilitate the learning of new tasks (e.g. new gestures and subjects).
• Figure 2: Examples of each of N-shot K-way meta training on the DVS Gestures dataset. The network is trained on K-shots of various classes (gestures) of the training dataset, meaning that $K$ recordings samples are used to learn an N-way classification task. This is repeated until convergence. Then, the meta-trained model is tested by training on $K$ samples of each unseen class and subject. For visualization purposes here, the images are DVS events averaged over $300$ms into frames, however the learning neural network operated at a time step of $1$ms for 100$ms$ samples.
• Figure 3: Pipeline for training the Intel Loihi on few-shot learning. (Left) The outer and inner objectives are first defined as differentiable functions of the neuron, synapse, and plasticity models. (Middle) These objectives are optimized using a Loihi simulation based on the Lava-DL library, which includes a functional simulator of the Loihi chip. Since Loihi uses integer precision weights, quantization-aware training is used where full precision shadow weights are quantized during the forward inference phase. Two gradient learning phases are performed, first in the inner loop (blue) and then in the outer loop with multiple shots of validation or test data (red). (Right) The resulting quantized parameters are transferred to the Intel Loihi neuromorphic hardware using Lava-DL NetX and Lava processes, enabling running on both the Loihi 1 and Loihi 2 in real time.
• Figure 4: A single neuron example of the SOEL learning rule. Given a Poisson spike input, SOEL learns to regulate the output of the neuron to a target firing rate within a time interval when the error of the neuron is 0. If the error is not 0 then learning is triggered every $t_{epoch}$ and plasticity adjusts the synaptic weight to correct the firing rate of the neuron to the desired rate. The green and orange dotted lines on the out spike plot indicate the threshold $\theta$