Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Plug-and-Play Homeostatic Spark: Zero-Cost Acceleration for SNN Training Across Paradigms

Rui Chen, Xingyu Chen, Yaoqing Hu, Shihan Kong, Zhiheng Wu, Junzhi Yu

TL;DR

This work tackles the instability and slow convergence of spiking neural network training by introducing AHSAR, a plug-and-play, parameter-free homeostatic controller. It combines rate sensing, a reaction-diffusion layer-wise homeostasis, and a global gain modulation to keep per-layer activity within a productive window across training paradigms. Across diverse architectures, modalities, and datasets, AHSAR yields faster convergence, higher accuracy, and improved out-of-distribution robustness without changing model structure or loss. The approach demonstrates that dual-timescale regulation—fast intra-epoch balancing and slow inter-epoch adaptation—is a simple, effective principle for scalable and efficient SNN training with minimal computational overhead.

Abstract

Spiking neural networks offer event driven computation, sparse activation, and hardware efficiency, yet training often converges slowly and lacks stability. We present Adaptive Homeostatic Spiking Activity Regulation (AHSAR), an extremely simple plug in and training paradigm agnostic method that stabilizes optimization and accelerates convergence without changing the model architecture, loss, or gradients. AHSAR introduces no trainable parameters. It maintains a per layer homeostatic state during the forward pass, maps centered firing rate deviations to threshold scales through a bounded nonlinearity, uses lightweight cross layer diffusion to avoid sharp imbalance, and applies a slow across epoch global gain that combines validation progress with activity energy to tune the operating point. The computational cost is negligible. Across diverse training methods, SNN architectures of different depths, widths, and temporal steps, and both RGB and DVS datasets, AHSAR consistently improves strong baselines and enhances out of distribution robustness. These results indicate that keeping layer activity within a moderate band is a simple and effective principle for scalable and efficient SNN training.

Plug-and-Play Homeostatic Spark: Zero-Cost Acceleration for SNN Training Across Paradigms

TL;DR

This work tackles the instability and slow convergence of spiking neural network training by introducing AHSAR, a plug-and-play, parameter-free homeostatic controller. It combines rate sensing, a reaction-diffusion layer-wise homeostasis, and a global gain modulation to keep per-layer activity within a productive window across training paradigms. Across diverse architectures, modalities, and datasets, AHSAR yields faster convergence, higher accuracy, and improved out-of-distribution robustness without changing model structure or loss. The approach demonstrates that dual-timescale regulation—fast intra-epoch balancing and slow inter-epoch adaptation—is a simple, effective principle for scalable and efficient SNN training with minimal computational overhead.

Abstract

Spiking neural networks offer event driven computation, sparse activation, and hardware efficiency, yet training often converges slowly and lacks stability. We present Adaptive Homeostatic Spiking Activity Regulation (AHSAR), an extremely simple plug in and training paradigm agnostic method that stabilizes optimization and accelerates convergence without changing the model architecture, loss, or gradients. AHSAR introduces no trainable parameters. It maintains a per layer homeostatic state during the forward pass, maps centered firing rate deviations to threshold scales through a bounded nonlinearity, uses lightweight cross layer diffusion to avoid sharp imbalance, and applies a slow across epoch global gain that combines validation progress with activity energy to tune the operating point. The computational cost is negligible. Across diverse training methods, SNN architectures of different depths, widths, and temporal steps, and both RGB and DVS datasets, AHSAR consistently improves strong baselines and enhances out of distribution robustness. These results indicate that keeping layer activity within a moderate band is a simple and effective principle for scalable and efficient SNN training.

Paper Structure

This paper contains 15 sections, 11 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Spiking Activity Analysis
  4. Method
  5. Rate sensing across physical and virtual time
  6. Layer wise reaction diffusion homeostasis
  7. Global gain modulation and integration
  8. Experiment
  9. Setup
  10. Quantitative Results
  11. On In-Distribution Data.
  12. On Out-of-Distribution Data.
  13. Qualitative Results
  14. Ablation Studies
  15. Conclusion

Figures (4)

  • Figure 1: Overview of current problem, biological inspiration and our method.
  • Figure 2: The impact of different scaling factors $S$ on model convergence under different training methods.
  • Figure 3: AHSAR overview. The Homeostatic controller senses layer-average spike rates, smooths and centers them. A diffusive term propagates corrections across neighboring layers via Laplacian smoothing, and the scaled thresholds are written back to each layer. The modules plug into a generic SNN over timesteps without changing the original forward/backward.
  • Figure 4: Spiking activity under different firing thresholds, where the "Scale factor" represents the scaling factor applied to the original firing threshold. $r_{AHSAR}$ denotes the scaling factor obtained using the AHSAR method.