Adaptive Synaptogenesis Implemented on a Nanomagnetic Platform

TL;DR

This work presents a brain-inspired model of adaptive synaptogenesis that combines supervised Hebbian learning with a hippocampal supervisor to form and prune distal synapses, thereby addressing the stability-plasticity dilemma for edge learning. The model uses running-average firing rates, a probabilistic synapse formation matrix, and shedding rules to maintain sparsity while enabling lifelong learning, validated through unsupervised and supervised simulations that show competitive accuracy with simple baselines like kNN. A hardware blueprint based on nanomagnetic devices—primarily straintronic MTJs and domain-wall synapses—outlines seven functional components (firing-rate measurement, comparator, analog multiplier/subtractor, non-volatile weights, and a tunable probability generator) to implement adaptive synaptogenesis on-chip with low power and non-volatile memory. The proposed straintronic hardware promises energy-efficient edge learning capabilities, with future work focusing on real-time demonstrations, device reliability under environmental conditions, and integration into embedded sensing and robotics applications.

Abstract

The human brain functions very differently from artificial neural networks (ANN) and possesses unique features that are absent in ANN. An important one among them is "adaptive synaptogenesis" that modifies synaptic weights when needed to avoid catastrophic forgetting and promote lifelong learning. The key aspect of this algorithm is supervised Hebbian learning, where weight modifications in the neocortex driven by temporal coincidence are further accepted or vetoed by an added control mechanism from the hippocampus during the training cycle, to make distant synaptic connections highly sparse and strategic. In this work, we discuss various algorithmic aspects of adaptive synaptogenesis tailored to edge computing, demonstrate its function using simulations, and design nanomagnetic hardware accelerators for specific functions of synaptogenesis.

Figures (14)

• Figure 1: (a) lower level representations of stripes and animal anatomy learnt by groups of neurons that will get wired together to (b) represent a higher level abstraction such as an animal species of zebra
• Figure 2: (a) Hebbian learning principle, $y_{1}$ firing just as inputs arrive at $x_1$ and $x_4$ implies weights $W_{11}$ and $W_{14}$ must be updated since Hebbian employs (b) spike timing dependent plasticity where a smaller $\Delta t$ causes larger change $\Delta W$.
• Figure 3: (a) Initial state where inputs are not stimulating $y_{1}$ enough, so its firing rate is low. (b) Synaptogenesis forms a new connection, increasing the firing rate above the threshold. Also, notice the smaller weights connected to $y_{2}$. (c) Low-value weights are 'shed,' and the firing rate remains unaffected, indicating they were not contributing to the activation of $y_{2}$.
• Figure 4: (a) Distribution of input dataset (500 points) run on (b) model with two outputs. (c) We monitor a few randomly selected weights over iterations (each iteration sends one datapoint across 60 epochs. (d) Pruned weight reduces firing rate which causes model to add weights until firing rate meets threshold
• Figure 5: (a) 1-D generated binary vectors where orthogonality ensures distinct classes. (b) Dataset used with 5 classes, variation in no. of datavectors (c) Model optimizes number of synapses after a few epochs (d) Sparse $W_{ij}$ matrix with higher number of points near the top since the associated inputs were excited the most.