Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neuron-centric Hebbian Learning

Andrea Ferigo, Elia Cunegatti, Giovanni Iacca

TL;DR

This work introduces Neuron-centric Hebbian Learning (NcHL), a biologically motivated approach that shifts Hebbian parameterization from synapses to neurons, reducing the parameter count from $5W$ to $5N$ and enabling scalable plasticity. A weightless variant (WNcHL) further lowers memory usage by approximating weights from recent neuron activations within a memory window. The authors formulate a neuron-centric update rule with per-neuron parameters and optimize it using Evolution Strategies across two robotic locomotion tasks, demonstrating comparable performance to traditional Hebbian learning while achieving orders-of-magnitude parameter savings. Comprehensive experiments reveal that NcHL and HL explore similar behavioral spaces, while WNcHL offers substantial memory savings with a controlled performance trade-off. This work suggests a promising path toward scalable, biologically plausible plasticity mechanisms in neural networks, with potential for extension to deep architectures and other high-dimensional tasks.

Abstract

One of the most striking capabilities behind the learning mechanisms of the brain is the adaptation, through structural and functional plasticity, of its synapses. While synapses have the fundamental role of transmitting information across the brain, several studies show that it is the neuron activations that produce changes on synapses. Yet, most plasticity models devised for artificial Neural Networks (NNs), e.g., the ABCD rule, focus on synapses, rather than neurons, therefore optimizing synaptic-specific Hebbian parameters. This approach, however, increases the complexity of the optimization process since each synapse is associated to multiple Hebbian parameters. To overcome this limitation, we propose a novel plasticity model, called Neuron-centric Hebbian Learning (NcHL), where optimization focuses on neuron- rather than synaptic-specific Hebbian parameters. Compared to the ABCD rule, NcHL reduces the parameters from $5W$ to $5N$, being $W$ and $N$ the number of weights and neurons, and usually $N \ll W$. We also devise a ``weightless'' NcHL model, which requires less memory by approximating the weights based on a record of neuron activations. Our experiments on two robotic locomotion tasks reveal that NcHL performs comparably to the ABCD rule, despite using up to $\sim97$ times less parameters, thus allowing for scalable plasticity

Neuron-centric Hebbian Learning

TL;DR

This work introduces Neuron-centric Hebbian Learning (NcHL), a biologically motivated approach that shifts Hebbian parameterization from synapses to neurons, reducing the parameter count from to and enabling scalable plasticity. A weightless variant (WNcHL) further lowers memory usage by approximating weights from recent neuron activations within a memory window. The authors formulate a neuron-centric update rule with per-neuron parameters and optimize it using Evolution Strategies across two robotic locomotion tasks, demonstrating comparable performance to traditional Hebbian learning while achieving orders-of-magnitude parameter savings. Comprehensive experiments reveal that NcHL and HL explore similar behavioral spaces, while WNcHL offers substantial memory savings with a controlled performance trade-off. This work suggests a promising path toward scalable, biologically plausible plasticity mechanisms in neural networks, with potential for extension to deep architectures and other high-dimensional tasks.

Abstract

One of the most striking capabilities behind the learning mechanisms of the brain is the adaptation, through structural and functional plasticity, of its synapses. While synapses have the fundamental role of transmitting information across the brain, several studies show that it is the neuron activations that produce changes on synapses. Yet, most plasticity models devised for artificial Neural Networks (NNs), e.g., the ABCD rule, focus on synapses, rather than neurons, therefore optimizing synaptic-specific Hebbian parameters. This approach, however, increases the complexity of the optimization process since each synapse is associated to multiple Hebbian parameters. To overcome this limitation, we propose a novel plasticity model, called Neuron-centric Hebbian Learning (NcHL), where optimization focuses on neuron- rather than synaptic-specific Hebbian parameters. Compared to the ABCD rule, NcHL reduces the parameters from to , being and the number of weights and neurons, and usually . We also devise a ``weightless'' NcHL model, which requires less memory by approximating the weights based on a record of neuron activations. Our experiments on two robotic locomotion tasks reveal that NcHL performs comparably to the ABCD rule, despite using up to times less parameters, thus allowing for scalable plasticity
Paper Structure (10 sections, 4 equations, 8 figures, 1 table)

This paper contains 10 sections, 4 equations, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and related work
  3. Methods
  4. Neuron-centric Hebbian Learning
  5. Neuron-centric Hebbian Learning "weightless"
  6. Evolution Strategies
  7. Experiments
  8. Experimental setup
  9. Experimental results
  10. Conclusions

Figures (8)

  • Figure 1: Number of parameters to optimize with HL and NcHL in Feed-Forward NN (toy example, 1 input and 1 output) when varying the no. of hidden layers and neurons therein.
  • Figure 2: RQ1: Distance trend of the best solution (avg. $\pm$ std. dev. across 10 runs) found at each generation by HL and NcHL in the various configurations of the VSR task. From top to bottom, the results refer to the High, Medium, and Low sensory configurations.
  • Figure 3: RQ1: Distribution of the best distance achieved across 10 runs by HL and NcHL in the various configurations of the VSR task.
  • Figure 4: RQ1: Distance trend of the best solution (avg. $\pm$ std. dev. across 10 runs) found at each generation by HL and NcHL in the two configurations of the Ant task.
  • Figure 5: RQ1: Distribution of the average distance in 100 rollouts of each best solution found in each of the 10 runs of HL and NcHL in the two configurations of the Ant task.
  • ...and 3 more figures