Heterosynaptic Circuits Are Universal Gradient Machines

TL;DR

The paper tackles how the brain can implement gradient-like learning while satisfying locality, flexibility, and robustness. It introduces a two-signal heterosynaptic rule with updates $\Delta \bar{V} = \eta p(t) \bar{h}^\top(t') - \gamma \bar{V}$ and $\Delta W = \eta p(t') \bar{h}^\top(t) - \gamma W$, and shows that when heterosynaptic stability (HS) and dynamical consistency (DC) hold, learning reduces to gradient descent with a PSD learning-rate $H = \bar{V}\bar{V}^\top$ and scalar consistency $\phi$. The theory is supported by simulations demonstrating gradient learning across diverse circuits, emergent Hebbian-like dynamics, and evolution-driven improvements in HSP alignment, including a root-node mechanism to fix update sign. Beyond biology, the framework suggests new optimizer architectures and scalable analog hardware that implement gradient descent without explicit gradient computation. Together, the work unifies Hebbian, heterosynaptic, meta-plasticity, and gradient-like learning, with significant implications for neuroscience, AI training, and neuromorphic hardware.

Abstract

We propose a design principle for the learning circuits of the biological brain. The principle states that almost any dendritic weights updated via heterosynaptic plasticity can implement a generalized and efficient class of gradient-based meta-learning. The theory suggests that a broad class of biologically plausible learning algorithms, together with the standard machine learning optimizers, can be grounded in heterosynaptic circuit motifs. This principle suggests that the phenomenology of (anti-) Hebbian (HBP) and heterosynaptic plasticity (HSP) may emerge from the same underlying dynamics, thus providing a unifying explanation. It also suggests an alternative perspective of neuroplasticity, where HSP is promoted to the primary learning and memory mechanism, and HBP is an emergent byproduct. We present simulations that show that (a) HSP can explain the metaplasticity of neurons, (b) HSP can explain the flexibility of the biology circuits, and (c) gradient learning can arise quickly from simple evolutionary dynamics that do not compute any explicit gradient. While our primary focus is on biology, the principle also implies a new approach to designing AI training algorithms and physically learnable AI hardware. Conceptually, our result demonstrates that contrary to the common belief, gradient computation may be extremely easy and common in nature.

Figures (15)

• Figure 1: Microscopic and macroscopic structures of biological heterosynaptic circuits (a-c) could implement gradient learning with the proposed "HSDC" mechanism (d). a: The minimal structure required to build a heterosynaptic circuit is a neuron with two incoming signals. Note that it does not require two inputs -- it could be a single axon that fires twice at different times. Due to its simple compositional nature, this circuit can be biologically realized across vastly different scales. b: Microscopically, the Purjinke cells (PC) in the mammalian cerebellum has a heterosynaptic motif, where parallel fiber (PF) carries the sensory input while climbing fiber (CF) carries learning or error signals. c: At a large scale, the bi-pathway structures between cortical regions can also implement heterosynaptic circuits. d: Heterosynaptic stability (HS) and dynamical consistency (DC) are sufficient to enable gradient learning. HS gives rise to local patches of neurons characterized by a scalar consistency score $\phi$; DC guarantees that $\phi$ has the same sign for all patches. For example, in c, the CF functions as heterosynaptic projections, which could give rise to $\phi$ values for the PC. In d, the feedback pathway from V1 to LGN can function as a heterosynaptic projection, which could give rise to consistency scores among the feedforward neurons or cortical columns in V1.
• Figure 2: Examples of heterosynaptic circuits for training a two-hidden-layer neural network nokland2016directlillicrap2016randomakrout2019deepkolen1994backpropagation. One can imagine a heterosynaptic circuit as a superposition of two graphs, one performing computation at time $t$ (solid) and the other performing computation at time $t'$ (dashed). In the figure, every node is a set of neurons, and every edge corresponds to a dense matrix sending connecting two such nodes. SGD can be seen as the special case of the KP algorithm when the two pathways are initialized symmetrically. Each edge can be either plastic or nonplastic. When nonplastic, the edges are fixed to be identity or random matrices. The rightmost (amorphous) shows an example of the most general type of heterosynaptic circuit to train such a network. Our theory shows that all the above circuits can simulate gradient learning.
• Figure 3: Meta-plasticity of neurons (a) and emergence of gradient learning in networks with microscopic random connectivity (b). a1: A neuron learning simple task with a root node $\tilde{h}$ and input node $\bar{h}$. a2: Plasticity of a synapse after it sees two samples (green curve shows 400 runs). The purple curve shows the plasticity of the synapse before it seem any data point. Prior experience alters the plasticity threshold for the synapse, consistent with biological observations abraham2008metaplasticity. b1: We train a four-hidden layer neuron network (two shown in the figure), where only connections between neighboring layers are allowed. Within the allowed connections, we randomly generete masks with $70\%$ density to simulate a different learning circuit topology. The figure illustrates two instances of such sampled connectivities. The color and style have the same meaning as in Figure \ref{['fig:circuit examples']}. b2: the learning trajectories across $2\times 10^4$ steps for $100$ i.i.d. randomly sampled learning circuits. Colors correspond to different layers: orange (1), blue (3), red (4). Gradient alignment is the correlation of the HSP update with the actual negative gradient. Hebbian overlap is the fraction of updates that have a positive alignment with the Hebbian update See Figure \ref{['fig:V stationarity and alignment']}-\ref{['fig:hebbian overlap zoom in']} for more detailed results.
• Figure 4: Training of a deep network with the nondifferentiable step-ReLU activation. Left: Illustration of the step-ReLU activation: $\sigma(x) = {\rm ReLU}(\lceil Qx \rceil) / Q$, a nondifferentiable approximate of ReLU. Mid: Performance of the model at different $Q$. In contrast, SGD can only train the last layer and cannot learn meaningful features from data. Right: gradient alignment to the ReLU network dual. As the theory predicts, the network self-assembles to a state that approximates the gradient of an approximate differentiable network, which explains the emergent learning behavior of the model. The colors denote layers 1 (orange), 2 (blue), and 3 (red), respectively.
• Figure 6: An evolutionary growth of heterosynaptic two-signal circuits. a: Examples of evolved circuits. The system consists of $4$ neurons whose target is to learn a linear regression problem, and the edges are pruned and grown according to a simple evolutionary algorithm for 200 generations. b: Example of an evolution trajectory for a densely initialized 100-neuron network. c: Results averaged over $400$ independent simulations, and the shaded region shows uncertainty. See Appendix \ref{['app sec: exp']} for details and another simulation with 100 neurons. The rows show 4-neuron evolution from a sparse init. (upper), 100-neuron evolution from a sparse init. (mid) and dense init (bottom). As evolution happens, the update rule of the circuit becomes increasingly aligned with the gradient.

Theorems & Definitions (17)

• Theorem 1
• proof
• Theorem 2
• Theorem 3
• proof
• Remark
• Definition 1
• Theorem 4
• proof
• Lemma 1