Learning-to-learn enables rapid learning with phase-change memory-based in-memory computing

TL;DR

The paper tackles the challenge of rapid, energy-efficient adaptation for edge AI by integrating learning-to-learn with PCM-based in-memory neuromorphic hardware. It demonstrates two complementary instantiations: (i) a CNN trained with MAML in software and deployed on NMHW for few-shot Omniglot classification, and (ii) a biologically inspired SNN trained with natural e-prop and deployed on NMHW to generate motor commands for a robotic arm, requiring only a single on-chip update. Key findings show that hardware-parity performance is achievable despite 4-bit PCM precision, with only a small subset of PCM devices updated per task, and meta-training can effectively occur in software without precise hardware models. Collectively, the work establishes a practical pathway for scalable, energy-efficient meta-learning on neuromorphic hardware and motivates software-then-hardware training pipelines for edge applications.

Abstract

There is a growing demand for low-power, autonomously learning artificial intelligence (AI) systems that can be applied at the edge and rapidly adapt to the specific situation at deployment site. However, current AI models struggle in such scenarios, often requiring extensive fine-tuning, computational resources, and data. In contrast, humans can effortlessly adjust to new tasks by transferring knowledge from related ones. The concept of learning-to-learn (L2L) mimics this process and enables AI models to rapidly adapt with only little computational effort and data. In-memory computing neuromorphic hardware (NMHW) is inspired by the brain's operating principles and mimics its physical co-location of memory and compute. In this work, we pair L2L with in-memory computing NMHW based on phase-change memory devices to build efficient AI models that can rapidly adapt to new tasks. We demonstrate the versatility of our approach in two scenarios: a convolutional neural network performing image classification and a biologically-inspired spiking neural network generating motor commands for a real robotic arm. Both models rapidly learn with few parameter updates. Deployed on the NMHW, they perform on-par with their software equivalents. Moreover, meta-training of these models can be performed in software with high-precision, alleviating the need for accurate hardware models.

Figures (10)

• Figure 1: Overview of learning-to-learn with neuromorphic hardware.a The general structure of meta-learning approaches used in this article. The inner loop learning is indicated by the gray box. The input to the inner loop is an initial parameter setting $\bm{\theta}$ and task inputs from a task $\mathcal{T}_i$. Based on these data points, a neural network model is updated $n$ times. In our settings, these updates were performed on a subset of the model parameters $\bm{\theta}$. The outer loop chooses in every iteration a new task $\mathcal{T}_i$ from the task family $\mathcal{F}(\mathcal{T})$, runs the inner loop, and updates the initial parameters $\bm{\theta}$ based on the errors in the inner loop. The goal is to find initial parameters $\bm{\theta}$ such that a few inner loop updates lead to good results on any task from $\mathcal{F}(\mathcal{T})$. b Unrolled meta-learning procedure that highlights the differences between task-specific adaptation of weights in the inner loop and the meta-parameters in the outer loop. c Schematic depiction of a phase-change memory device and its inner working. Information is stored in the phase configuration of the material and electrical pulses can be used to switch between the amorphous and the crystalline phase. d The employed neuromorphic hardware comprises a crossbar array structure where at each intersection four PCM devices (4R) and eight control transistors (8T) are located. Two PCM devices represent positive weights, bitline positive (BL$^+$), and two represent the negative weights, bitline negative (BL$^-$). The weights of a neural network are then mapped onto the crossbar structure and network inputs are provided to the positive devices using (WL$^+$) and to the negative devices using (WL$^-$).
• Figure 1: Detailed analysis of dense layer.a Histogram and evolution of the normalized weights of the dense layer. The normalized weights are clustered into five bins, e.g., from $0.$ to $0.2$, from $0.2$ to $0.4$, etc. The individual bars within the bins show the weights before any update (blue bar), after the first update (orange bar), after the second update (green bar), after the third update (purple bar) and after the fourth update (red bar). b Cumulative distribution of the weights within the individual bins. While the weight values within the first few bins remain rather unchanged, the weight distribution within the last two bins changes significantly over the course of the updates.
• Figure 2: (Caption on next page.)
• Figure 2: Evaluation of a second trajectory for the robotic task described in Section \ref{['sec:results_robotic_arm_control']}. a Angular velocities and trajectories in the Euclidean space of the meta-trained network in software (blue) with NMHW (orange) before the inner loop update. b Angular velocities and trajectories in the Euclidean space of the networks after one-shot learning. The green trajectory shows the trajectory of the ED-Scorbot robot.
• Figure 3: Few-shot image classification on Omniglot with MAML.a Illustration of the inner and outer loops in the MAML setup. In the inner loop, a software model was used for meta-training. The evaluation was performed both in software and in neuromorphic hardware. For the inner loop training, we performed four gradient updates. b Schematic depiction of the movement in parameter space during MAML. The initial parameters $\bm{\theta}$ are optimized in the outer loop (bold trajectory) and the inner loop performs four task-specific adaptation steps (small arrows) such that the model achieves high classification accuracy. c Illustration of the input data from the Omniglot dataset for the 5-way 5-shot classification task on the left and the corresponding ground-truth targets on the right. A typical evolution of the classification performance of the model in the inner loop is illustrated in the middle. d Architecture of the four layer convolutional neural network with a dense layer on top that is employed to solve the classification task. e Schematic depiction of the mapping of the neural network to the NMHW. The convolutional layers are split into two parts and spread across the two crossbar arrays of the NMWH. f Evolution of the loss during outer loop training of a 4 bit (orange) and a 32 bit (blue) model in software. g Classification accuracy of the various models on $100$ new unseen tasks. h Classification accuracy of the of the various models during inner loop training.