Sleep-Based Homeostatic Regularization for Stabilizing Spike-Timing-Dependent Plasticity in Recurrent Spiking Neural Networks

TL;DR

Spike-timing-dependent plasticity in recurrent spiking neural networks can produce unstable weight dynamics, including saturation and forgetting. The authors introduce a sleep-based homeostatic regularization that interleaves wake learning with sleep phases in which inputs are suppressed and weights decay toward a homeostatic baseline, with intrinsic noise enabling replay-like consolidation. In MNIST-family benchmarks, moderate sleep (10–20%) improves stability and performance for the STDP-SNN, while surrogate-gradient SNNs show no systematic gains from sleep, highlighting the locality-specific benefits of this approach. The findings suggest sleep-like renormalization as a practical, hardware-friendly mechanism to stabilize local Hebbian learning in neuromorphic systems, albeit with careful tuning and limited applicability to non-local learning rules. Overall, the work provides mechanistic insight into how sleep-inspired processes can support long-term stability and generalization in Hebbian SNNs, paving the way for robust on-chip learning.

Abstract

Spike-timing-dependent plasticity (STDP) provides a biologically-plausible learning mechanism for spiking neural networks (SNNs); however, Hebbian weight updates in architectures with recurrent connections suffer from pathological weight dynamics: unbounded growth, catastrophic forgetting, and loss of representational diversity. We propose a neuromorphic regularization scheme inspired by the synaptic homeostasis hypothesis: periodic offline phases during which external inputs are suppressed, synaptic weights undergo stochastic decay toward a homeostatic baseline, and spontaneous activity enables memory consolidation. We demonstrate that this sleep-wake cycle prevents weight saturation while preserving learned structure. Empirically, we find that low to intermediate sleep durations (10-20\% of training) improve stability on MNIST-like benchmarks in our STDP-SNN model, without any data-specific hyperparameter tuning. In contrast, the same sleep intervention yields no measurable benefit for the surrogate-gradient spiking neural network (SG-SNN). Taken together, these results suggest that periodic, sleep-based renormalization may represent a fundamental mechanism for stabilizing local Hebbian learning in neuromorphic systems, while also indicating that special care is required when integrating such protocols with existing gradient-based optimization methods.

Figures (9)

• Figure 1: Network architecture: three-layer SNN with feedforward input ($P=10\%$), recurrent excitation ($P=15\%$), and lateral inhibition ($P=25\%$). Sparse connectivity enables event-driven computation.
• Figure 2: STDP learning window: weights strengthen (positive $\Delta t$) when presynaptic spike precedes postsynaptic, weaken (negative $\Delta t$) otherwise.
• Figure 3: Weight dynamics: wake phases show deterministic linear STDP-driven growth; sleep phases show power-law regularization, Eq. \ref{['eq:decay']} with superimposed random noise resembling realizations of Gaussian random variables, representing spontaneous neural activity during consolidation. System cycles robustly between wake (input-driven) and sleep (homeostatic noisy decay), maintaining weights within plausible intervals.
• Figure 4: (a) training set composed of four geometric classes (triangle, circle, square, and cross), each subjected to Gaussian noise $\mathcal{N}(0,0.02)$. All training examples use the same base shapes, with noise as the stochastic component introducing sample-by-sample variance. (b) shows accuracy attained with the no-sleep and sleep protocol.
• Figure 5: (a) denotes average excitatory and inhibitory weights across batches training with sleep regularization. The grey area capture min and max weights per batch. (b) denotes the same, but without sleep to stabilize learning. (c) shows weight changes across a single batch training session with the sleep protocol active. Along the x-axis, we plot time in milliseconds, and along the y-axis, we plot change in weights. Similar to (d), stippled lines represent inhibitory weights and solid lines excitatory min/max and mean weights.