Weight transport through spike timing for robust local gradients

TL;DR

Weight transport in physical neural substrates challenges non-local gradient methods. The paper introduces Spike-based Alignment Learning (SAL), a fully local, spike-timing–driven rule that uses noise to extract and correct asymmetries between reciprocal connections, aligning effective forward and backward pathways. SAL is analytically grounded and demonstrated in two domains: (1) spiking sampling networks pursuing target Boltzmann distributions, where SAL improves convergence under synaptic and plasticity noise, and (2) cortical microcircuit models enabling biologically plausible, locally implemented backpropagation through aligned feedback. The results show robust weight symmetrization, improved learning with noise, and a viable path toward hardware-friendly, on-chip learning in neuromorphic systems and biologically plausible gradient-based networks.

Abstract

In both machine learning and in computational neuroscience, plasticity in functional neural networks is frequently expressed as gradient descent on a cost. Often, this imposes symmetry constraints that are difficult to reconcile with local computation, as is required for biological networks or neuromorphic hardware. For example, wake-sleep learning in networks characterized by Boltzmann distributions builds on the assumption of symmetric connectivity. Similarly, the error backpropagation algorithm is notoriously plagued by the weight transport problem between the representation and the error stream. Existing solutions such as feedback alignment tend to circumvent the problem by deferring to the robustness of these algorithms to weight asymmetry. However, they are known to scale poorly with network size and depth. We introduce spike-based alignment learning (SAL), a complementary learning rule for spiking neural networks, which uses spike timing statistics to extract and correct the asymmetry between effective reciprocal connections. Apart from being spike-based and fully local, our proposed mechanism takes advantage of noise. Based on an interplay between Hebbian and anti-Hebbian plasticity, synapses can thereby recover the true local gradient. This also alleviates discrepancies that arise from neuron and synapse variability -- an omnipresent property of physical neuronal networks. We demonstrate the efficacy of our mechanism using different spiking network models. First, we show how SAL can significantly improve convergence to the target distribution in probabilistic spiking networks as compared to Hebbian plasticity alone. Second, in neuronal hierarchies based on cortical microcircuits, we show how our proposed mechanism effectively enables the alignment of feedback weights to the forward pathway, thus allowing the backpropagation of correct feedback errors.

Figures (8)

• Figure 1: The weight transport problem in physical neuronal networks. Many successful learning rules presuppose certain symmetry properties of the underlying network structure. This poses challenges to reconcile them with the locality principle of physical neuronal computation. a1)⁠ In wake-sleep learning, the network is trained using correlation measurements. Since these measurements are carried out by two distinct, individually parameterized synapses (here $W_{ji}$ and $W_{ij}$), weight updates are not symmetric. a2)⁠ The algorithm relies on the transportation of the gradient across layers, which requires a copy of the forward weights (green) to the backward path (blue). b)⁠ Depending on the misalignment between the true gradient (green arrow) and the trajectory followed by , learning can be slowed down significantly (violet) or fail completely (orange).
• Figure 2: Principles of spike-based alignment learning.a)⁠ Two reciprocally connected neurons $i$ and $j$ embedded in a recurrent neural network fire stochastically due to external spike noise. Here, weight $W_{ji}$ (orange synapse) is stronger than $W_{ij}$ (green). Because symmetrizes effective weights rather than pure synaptic strengths, effects of morphological differences between the two neurons or the synaptic location at the dendritic tree are implicitly balanced out. This underpins the feasibility of in both biological and neuromorphic substrates (illustration adapted from muller2020extending). b)⁠ Example spike trains with colored spike-timing differences $\Delta t$ (causal in purple and anti-causal in blue), as seen from the perspective of $W_{ji}$. c1)⁠ distribution for two reciprocally connected neurons as described in \ref{['sec:results:sal_spiketiming']}. The upper panel shows two excitatory weights, the lower two inhibitory ones. c2)⁠ with an anti-Hebbian window used by for weight alignment. d)⁠ Time course of symmetrization with . If both weights are plastic, they converge to their common mean (red); if only one weight uses , it converges to the other one (gray). e)⁠ Phase diagram showing the evolution of the two weights under . The arrows indicate the direction of the weight update through , the color map in the background the magnitude of the update. The blue line indicates the attractor of which lies on the diagonal $W_{ij} = W_{ji}$. always converges to the desired solution $W_{ij} = W_{ji}$ from any starting point in the $W_{ij}$-$W_{ji}$-plane. The example trajectories from d) are depicted in red and gray.
• Figure 3: Working principle of .a)⁠ An consists of a recurrent spiking network with symmetric reciprocal connections in theory, but asymmetric ones in practice. b)⁠ Each neuron fires stochastically as a function of the membrane potential (blue), which gives rise to a sampling process from an underlying distribution $p(\boldsymbol{z})$. The refractory state of each neuron is mapped to a binary variable $z \in [0, 1]$ (green and red). with a left-right symmetric window (orange) is used to implement spike-based wake-sleep. Because the synaptic update is local to each synapse, reciprocal weight updates are also asymmetric. c)⁠ Example sampled distribution for $N=4$ neurons.
• Figure 4: Synaptic noise in . An is initialized with a symmetric weight matrix and trained with pure -based wake-sleep (blue curve), which serves as a baseline (no noise). We also train a noised weight matrix (additive Gaussian noise with standard deviation $\sigma^\mathrm{noise}_\mathrm{init}$) with (green) and without (orange) a phase in addition to -based wake-sleep learning. a1)⁠ Evolution of the variance of weight differences $W_{ij} - W_{ji}$ as a measure for the weight asymmetry for the three cases after each epoch. a2)⁠ Variance of the weight differences after training. b1)⁠ Evolution of the during training between the freely sampled model distribution $p$ and the target distribution $p^*$. b2)⁠ Final after training as a function of the initial noise variance.
• Figure 6: enables accurate in a spiking cortical microcircuit model.a)⁠ Cortical microcircuit model for biologically plausible error backpropagation based on sacramento2018dendritic, which we augment with a spiking mechanism. b1)⁠ Performance of different learning rules on a teacher mimic task. outperforms , and training performance is on par with weight copying (). Note the large performance variability of , which depends critically on the initial weights. b2)⁠ Evolution of top-down and bottom-up weights during learning for a case where initial feedback weights carry the wrong sign. Forward weights $\boldsymbol{W}^\mathrm{PP}_{1,0}$ first evolve in the wrong direction (with , they would explode, see h), but the gradual alignment of the backward weights $\boldsymbol{B}^\mathrm{PP}_{1, 2}$ improves the feedback error signals, ultimately recovering the performance of vanilla BP. c)⁠ Distribution of final validation loss and learned weights in the student network.