Spike-timing-dependent Hebbian learning as noisy gradient descent

TL;DR

The paper analyzes a biologically plausible spike-timing-dependent plasticity (STDP) rule in a d-input, 1-output network by recasting it as a noisy gradient descent on a non-convex loss L on the probability simplex. It proves that, starting from a strictly dominant input, the output aligns with the strongest input at exponential (linear) rate on an event of high probability, even in the presence of constant noise. The results connect STDP to entropic/mirror descent on the simplex via the Shahshahani metric and provide a gradient-flow analogue with explicit convergence bounds. Extensions to multiple read-out neurons, time-inhomogeneous inputs, and correlated inputs illustrate broader applicability and reveal connections to replicator dynamics and PCA-like objectives, while acknowledging biological and mathematical limitations such as weight explosion and small-network assumptions.

Abstract

Hebbian learning is a key principle underlying learning in biological neural networks. We relate a Hebbian spike-timing-dependent plasticity rule to noisy gradient descent with respect to a non-convex loss function on the probability simplex. Despite the constant injection of noise and the non-convexity of the underlying optimization problem, one can rigorously prove that the considered Hebbian learning dynamic identifies the presynaptic neuron with the highest activity and that the convergence is exponentially fast in the number of iterations. This is non-standard and surprising as typically noisy gradient descent with fixed noise level only converges to a stationary regime where the noise causes the dynamic to fluctuate around a minimiser.

Figures (6)

• Figure 1: Neural network with a single output neuron
• Figure 2: Contour plot of the loss function $L$ from Eq. (\ref{['eq:LossFunction']}) on the probability simplex $\mathfrak{P}$ for $d=3$ with different overlays. Left: Three sample trajectories of Eq. (\ref{['eq:LearningRule_P']}) with different initial configurations $\mathbf{p}(0)$. Middle: Stream plot of the gradient field given by Eq. (\ref{['eq:Lossgradient']}). Right: 100 sample trajectories of Eq. (\ref{['eq:LearningRule_P']}) with $\mathbf{p}(0)= (0.3,0.3,0.4)^\top$. All trajectories are simulated with 2000 iteration steps, learning rate $\alpha=0.01$ and $\mathbf{Z}(k)\sim \operatorname{Unif}([-1,1]^d)$.
• Figure 3: Considered biological neural network with spike trains and membrane potential $Y_t$ of the postsynaptic neuron.
• Figure 4: Neural network with $d$ input/output neurons.
• Figure 5: Probability matrix $\mathbf{P}(k)$ arising from the weight dynamic $\mathbf{W}(k)$ of Algorithm \ref{['alg:MultipleWeights']} for dimensions $n=d=3$. The weights are initialised equally, and the intensities are given by $\boldsymbol{\lambda}=(10,7.5,5)^\top$. The resulting initial probabilities are $\mathbf{p}_1(0)=\mathbf{p}_2(0)=\mathbf{p}_3(0) =(4/9, 1/3, 2/9)^\top$. Left: A single trajectory with learning rates $10^{-3}(1,0.75,0.5)^\top$ and $4\times 10^{4}$ iterations. The markers $\times$ and $\bullet$ correspond to the probabilities at $k=4\times 10^{3}$ and $k=10^4$. Middle & right: The Frobenius error $\Vert \mathbf{P}(k) - \mathbb{I}\Vert^2/2$ of 100 trajectories with learning rates $10^{-3}(1,0.75,0.5)^\top$ and $10^{-4}(1,0.75,0.5)^\top$, respectively.

Theorems & Definitions (33)

• Lemma 2.1
• Theorem 2.2
• Remark 2.3
• Theorem 2.4
• proof : Proof of Lemma \ref{['lem:Losslandscape']}
• Remark A.1
• Lemma A.2
• proof
• Proposition A.3
• proof