Hebbian Attractor Networks for Robot Locomotion

Abstract

Biological neural networks continuously adapt and modify themselves in response to experiences throughout their lifetime - a capability largely absent in artificial neural networks. Hebbian plasticity offers a promising path toward rapid adaptation in changing environments. Here, we introduce Hebbian Attractor Networks (HAN), a class of plastic neural networks in which local weight update normalization induces emergent attractor dynamics. Unlike prior approaches, HANs employ dual-timescale plasticity and temporal averaging of pre- and postsynaptic activations to induce either co-dynamic limit cycles or fixed-point weight attractors. Using simulated locomotion benchmarks, we gain insight into how Hebbian update frequency and activation averaging influence weight dynamics and control performance. Our results show that slower updates, combined with averaged pre- and postsynaptic activations, promote convergence to stable weight configurations, while faster updates yield oscillatory co-dynamic systems. We further demonstrate that these findings generalize to high-dimensional quadrupedal locomotion with a simulated Unitree Go1 robot. These results highlight how the timing of plasticity shapes neural dynamics in embodied systems, providing a principled characterization of the attractor regimes that emerge in self-modifying networks.

Figures (22)

• Figure 1: Hebbian neural networks are updated during the lifetime based on correlations between pre- and postsynaptic activations and do not rely on gradient information. By introducing a slower rate of Hebbian updates than the action inference or averaging of the synaptic history, we can induce different attractor dynamics in weight space.
• Figure 2: Dual-timescale structure of HNN. Hebbian updates are executed at a lower frequency than the forward pass (here, $\tau_{\text{hebb}} = 3\times \tau_{\text{NN}}$), and the moving average of pre- and postsynaptic activations is applied (here, $M=2$).
• Figure 3: Analysis of weight dynamics under three Hebbian learning conditions (A, B, E). The first row shows trajectories of 10 randomly selected plastic weights over time. The second row presents the total network plasticity, quantified as the summed $\ell_2$-norm of weight changes at each timestep. The third row depicts the evolution of plastic weights projected onto the first two principal components via Principal component analysis (PCA).
• Figure 4: The non-zero frequencies of action and weight signals exhibit peaks at approximately 4Hz and 8Hz. Interrupting the Hebbian update causes a pronounced drop in step rewards during deployment for the HAN condition (B). The HNN in condition (E) shows a minor effect in step rewards.
• Figure 5: Fitness at 1000 generations with different $f_{\text{hebb}}$ and $M$ ($n=16$). A window length of $M=1$ corresponds to no moving average, and $f_{\text{hebb}}=20$ corresponds to no reduction in update frequency.