Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hebbian Learning with Global Direction

Wenjia Hua, Kejie Zhao, Luziwei Leng, Ran Cheng, Yuxin Ma, Qinghai Guo

TL;DR

The paper tackles the scalability and biological plausibility gaps of backpropagation by introducing Global-guided Hebbian Learning (GHL), which fuses local Hebbian updates with a global, sign-based modulation derived from the task gradient. It formulates a three-factor learning rule where a local Hebbian term is modulated by a global signal, enabling end-to-end training across diverse architectures without full gradient propagation. The local component uses competitive learning with Oja's rule, while the global component provides directional guidance, leading to competitive performance on CIFAR-10/100 and ImageNet, and demonstrated scalability to very deep networks. This work advances bio-inspired learning by showing that combining local plasticity with global task information can closely approach backpropagation performance and scale to large datasets and architectures, with potential applications in neuromorphic hardware and beyond.

Abstract

Backpropagation algorithm has driven the remarkable success of deep neural networks, but its lack of biological plausibility and high computational costs have motivated the ongoing search for alternative training methods. Hebbian learning has attracted considerable interest as a biologically plausible alternative to backpropagation. Nevertheless, its exclusive reliance on local information, without consideration of global task objectives, fundamentally limits its scalability. Inspired by the biological synergy between neuromodulators and local plasticity, we introduce a novel model-agnostic Global-guided Hebbian Learning (GHL) framework, which seamlessly integrates local and global information to scale up across diverse networks and tasks. In specific, the local component employs Oja's rule with competitive learning to ensure stable and effective local updates. Meanwhile, the global component introduces a sign-based signal that guides the direction of local Hebbian plasticity updates. Extensive experiments demonstrate that our method consistently outperforms existing Hebbian approaches. Notably, on large-scale network and complex datasets like ImageNet, our framework achieves the competitive results and significantly narrows the gap with standard backpropagation.

Hebbian Learning with Global Direction

TL;DR

The paper tackles the scalability and biological plausibility gaps of backpropagation by introducing Global-guided Hebbian Learning (GHL), which fuses local Hebbian updates with a global, sign-based modulation derived from the task gradient. It formulates a three-factor learning rule where a local Hebbian term is modulated by a global signal, enabling end-to-end training across diverse architectures without full gradient propagation. The local component uses competitive learning with Oja's rule, while the global component provides directional guidance, leading to competitive performance on CIFAR-10/100 and ImageNet, and demonstrated scalability to very deep networks. This work advances bio-inspired learning by showing that combining local plasticity with global task information can closely approach backpropagation performance and scale to large datasets and architectures, with potential applications in neuromorphic hardware and beyond.

Abstract

Backpropagation algorithm has driven the remarkable success of deep neural networks, but its lack of biological plausibility and high computational costs have motivated the ongoing search for alternative training methods. Hebbian learning has attracted considerable interest as a biologically plausible alternative to backpropagation. Nevertheless, its exclusive reliance on local information, without consideration of global task objectives, fundamentally limits its scalability. Inspired by the biological synergy between neuromodulators and local plasticity, we introduce a novel model-agnostic Global-guided Hebbian Learning (GHL) framework, which seamlessly integrates local and global information to scale up across diverse networks and tasks. In specific, the local component employs Oja's rule with competitive learning to ensure stable and effective local updates. Meanwhile, the global component introduces a sign-based signal that guides the direction of local Hebbian plasticity updates. Extensive experiments demonstrate that our method consistently outperforms existing Hebbian approaches. Notably, on large-scale network and complex datasets like ImageNet, our framework achieves the competitive results and significantly narrows the gap with standard backpropagation.
Paper Structure (11 sections, 5 equations, 3 figures, 5 tables, 1 algorithm)

This paper contains 11 sections, 5 equations, 3 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Method
  3. Overview of GHL framework
  4. Competitive Hebbian learning
  5. Global direction as a third factor
  6. Experiments and Results
  7. Comparison with existing Hebbian methods on CIFAR-10 and CIFAR-100 datasets
  8. Scalability on complex datasets and deeper networks
  9. The architecture’s impact
  10. Ablation on local information
  11. Discussion

Figures (3)

  • Figure 1: Comparison of different training methods.Left: Standard BP training process, using global error signals to compute precise gradients for weight updates. Middle: Two-factor Hebbian learning approach, which relies only on local information from pre and post synaptic neurons to update weights, without considering global task objectives. Right: Our method, Global-guided Hebbian Learning (GHL), integrates local Hebbian plasticity with a global sign-based modulation.
  • Figure 2: Performance comparison of different architectures using SoftHebb and GHL. The x-axis represents network depth (the number of convolutional layers), and the y-axis shows Top-1 accuracy. The architectures tested have initial channel counts of 96, which are then expanded by factors of 1, 2, and 4, respectively.
  • Figure : GHL Training