A Backpropagation-Free Feedback-Hebbian Network for Continual Learning Dynamics

TL;DR

The paper asks whether a compact, backpropagation-free feedback–Hebbian network can express interpretable continual-learning dynamics. It introduces a two-forward, two-feedback architecture trained with a local rule combining centered Hebbian covariance, Oja-style stabilization, and a local supervised drive, and analyzes sequential (A→B) and interleaved (A,B) training schedules on a two-pair association task. Key findings include LTD-like unlearning at the forward output with retention of A in the feedback pathway during sequential training, concurrent co-maintenance of both associations under temporal alternation, and clear evidence that a 2F+2B architecture is minimal yet sufficient, with dedicated feedback necessary for regeneration and conditioning. Ablations show that covariance, normalization, and supervision have distinct roles in shaping synaptic dynamics, supporting a mechanistic division of labor where forward layers implement the current mapping while feedback preserves and propagates association traces. This work demonstrates that local, feedback-trained architectures can realize continual-learning primitives with interpretable, circuit-level dynamics, offering a biologically plausible complement to gradient-based methods.

Abstract

Feedback-rich neural architectures can regenerate earlier representations and inject temporal context, making them a natural setting for strictly local synaptic plasticity. We ask whether a minimal, backpropagation-free feedback--Hebbian system can already express interpretable continual-learning--relevant behaviors under controlled training schedules. We introduce a compact prediction--reconstruction architecture with two feedforward layers for supervised association learning and two dedicated feedback layers trained to reconstruct earlier activity and re-inject it as additive temporal context. All synapses are updated by a unified local rule combining centered Hebbian covariance, Oja-style stabilization, and a local supervised drive where targets are available, requiring no weight transport or global error backpropagation. On a small two-pair association task, we characterize learning through layer-wise activity snapshots, connectivity trajectories (row/column means of learned weights), and a normalized retention index across phases. Under sequential A->B training, forward output connectivity exhibits a long-term depression (LTD)-like suppression of the earlier association while feedback connectivity preserves an A-related trace during acquisition of B. Under deterministic interleaving A,B,A,B,..., both associations are concurrently maintained rather than sequentially suppressed. Architectural controls and rule-term ablations isolate the role of dedicated feedback in regeneration and co-maintenance, and the role of the local supervised term in output selectivity and unlearning. Together, the results show that a compact feedback pathway trained with local plasticity can support regeneration and continual-learning--relevant dynamics in a minimal, mechanistically transparent setting.

Figures (10)

• Figure 1: Backpropagation–free feedback–Hebbian network. Green: feedforward pathways; blue: dedicated feedback pathways. Feedforward and feedback use distinct weight matrices; backward signals do not reuse feedforward weights in reverse.
• Figure 2: Baseline forward prediction for pair A (input site 3 active). Output activity across the 10 sites (a) before training and (b) after Phase A. Target output sites are 8 and 9.
• Figure 3: Baseline feedback-driven regeneration for pair A. Regenerated input-side activity across the 10 sites when driven by the pair A output pattern (a) before training and (b) after Phase A. The corresponding stimulated input site is 3.
• Figure 4: Sequential A$\rightarrow$B training (Regime I): forward layer 2 output connectivity across epochs. Traces for the pair B target outputs (sites 5–6) and the earlier pair A targets (sites 8–9) are emphasized; other sites are shown faintly. The Phase A/Phase B transition and the baseline/end-of-phase epochs ($e{=}0,10,20$) are marked.
• Figure 5: Sequential A$\rightarrow$B training (Regime I): feedback layer 2 input connectivity across epochs. Traces corresponding to forward-output sites involved in pair A (sites 8–9) are emphasized to assess retention during Phase B; other sites are shown faintly. The Phase A/Phase B transition and the baseline/end-of-phase epochs ($e{=}0,10,20$) are marked.