Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Delays in Spiking Neural Networks: A State Space Model Approach

Sanja Karilanova, Subhrakanti Dey, Ayça Özçelikkale

TL;DR

This work presents a principled state-space delay mechanism for spiking neural networks, enabling neurons to access a finite history of inputs through an auxiliary memory state while remaining compatible with common neuron models like LIF and adLIF. By integrating a time-shift state transition and a delay-coupling term, the framework provides a differentiable, architecture-friendly means to inject temporal memory, with careful analysis of delay order, delay-parameter settings, and training overhead. Empirical evaluation on the SHD dataset shows competitive accuracy with existing delay-based SNN approaches, with notable gains in small networks and modest increases in training time and parameter count. The study highlights the practical viability of hardware-friendly delayed SNNs and points to richer neuron models and hardware implementations as promising directions for future work.

Abstract

Spiking neural networks (SNNs) are biologically inspired, event-driven models that are suitable for processing temporal data and offer energy-efficient computation when implemented on neuromorphic hardware. In SNNs, richer neuronal dynamic allows capturing more complex temporal dependencies, with delays playing a crucial role by allowing past inputs to directly influence present spiking behavior. We propose a general framework for incorporating delays into SNNs through additional state variables. The proposed mechanism enables each neuron to access a finite temporal input history. The framework is agnostic to neuron models and hence can be seamlessly integrated into standard spiking neuron models such as LIF and adLIF. We analyze how the duration of the delays and the learnable parameters associated with them affect the performance. We investigate the trade-offs in the network architecture due to additional state variables introduced by the delay mechanism. Experiments on the Spiking Heidelberg Digits (SHD) dataset show that the proposed mechanism matches the performance of existing delay-based SNNs while remaining computationally efficient. Moreover, the results illustrate that the incorporation of delays may substantially improve performance in smaller networks.

Delays in Spiking Neural Networks: A State Space Model Approach

TL;DR

This work presents a principled state-space delay mechanism for spiking neural networks, enabling neurons to access a finite history of inputs through an auxiliary memory state while remaining compatible with common neuron models like LIF and adLIF. By integrating a time-shift state transition and a delay-coupling term, the framework provides a differentiable, architecture-friendly means to inject temporal memory, with careful analysis of delay order, delay-parameter settings, and training overhead. Empirical evaluation on the SHD dataset shows competitive accuracy with existing delay-based SNN approaches, with notable gains in small networks and modest increases in training time and parameter count. The study highlights the practical viability of hardware-friendly delayed SNNs and points to richer neuron models and hardware implementations as promising directions for future work.

Abstract

Spiking neural networks (SNNs) are biologically inspired, event-driven models that are suitable for processing temporal data and offer energy-efficient computation when implemented on neuromorphic hardware. In SNNs, richer neuronal dynamic allows capturing more complex temporal dependencies, with delays playing a crucial role by allowing past inputs to directly influence present spiking behavior. We propose a general framework for incorporating delays into SNNs through additional state variables. The proposed mechanism enables each neuron to access a finite temporal input history. The framework is agnostic to neuron models and hence can be seamlessly integrated into standard spiking neuron models such as LIF and adLIF. We analyze how the duration of the delays and the learnable parameters associated with them affect the performance. We investigate the trade-offs in the network architecture due to additional state variables introduced by the delay mechanism. Experiments on the Spiking Heidelberg Digits (SHD) dataset show that the proposed mechanism matches the performance of existing delay-based SNNs while remaining computationally efficient. Moreover, the results illustrate that the incorporation of delays may substantially improve performance in smaller networks.

Paper Structure

This paper contains 29 sections, 9 equations, 2 figures, 5 tables.

Table of Contents

  1. Abstract
  2. Introduction
  3. Related Work
  4. Methods
  5. Preliminaries
  6. General Spiking Neuron Model
  7. State Space Model with Time Shift State Transition Matrix
  8. Proposed Model
  9. LIF and RLIF example
  10. adLIF and RadLIF example
  11. Delay parameters
  12. Computational complexity
  13. Number of trainable parameters
  14. Memory
  15. Experimental Set-up
  16. ...and 14 more sections

Figures (2)

  • Figure 1: Illustration of different approaches of modeling delays. Our proposed delays are modeled within the state dynamics of the neuron (green). Figure is inspired by DelGrad.
  • Figure 2: High-level illustration of the proposed neuron model. The figure shows how the delay components are incorporated into the traditional SNN neuron model (LIF neuron shown).