Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Latency Coding Framework for Deep Spiking Neural Networks with Ultra-Low Latency

Yi Lu, Jianhao Ding, Zhaofei Yu

Abstract

Spiking neural networks (SNNs) offer a biologically inspired computing paradigm with significant potential for energy-efficient neural processing. Among neural coding schemes of SNNs, Time-To-First-Spike (TTFS) coding, which encodes information through the precise timing of a neuron's first spike, provides exceptional energy efficiency and biological plausibility. Despite its theoretical advantages, existing TTFS models lack efficient training methods, suffering from high inference latency and limited performance. In this work, we present a comprehensive framework, which enables the efficient training of deep TTFS-coded SNNs by employing backpropagation throuh time (BPTT) algorithm. We name the generalized TTFS coding method in our framework as latency coding. The framework includes: (1) a latency encoding (LE) module with feature extraction and straight-through estimators to address severe information loss in direct intensity-to-latency mapping and ensure smooth gradient flow; (2) relaxation of the strict single-spike constraint of traditional TTFS, allowing neurons of intermediate layers to fire multiple times to mitigating gradient vanishing in deep networks; (3) a temporal adaptive decision (TAD) loss function that dynamically weights supervision signals based on sample-dependent confidence, resolving the incompatibility between latency coding and standard cross-entropy loss. Experimental results demonstrate that our method achieves state-of-the-art accuracy in comparison to existing TTFS-coded SNNs with ultra-low inference latency and superior energy efficiency. The framework also demonstrates improved robustness against input corruptions. Our study investigates the characteristics and potential of latency coding in scenarios demanding rapid response, providing valuable insights for further exploiting the temporal learning capabilities of SNNs.

A Latency Coding Framework for Deep Spiking Neural Networks with Ultra-Low Latency

Abstract

Spiking neural networks (SNNs) offer a biologically inspired computing paradigm with significant potential for energy-efficient neural processing. Among neural coding schemes of SNNs, Time-To-First-Spike (TTFS) coding, which encodes information through the precise timing of a neuron's first spike, provides exceptional energy efficiency and biological plausibility. Despite its theoretical advantages, existing TTFS models lack efficient training methods, suffering from high inference latency and limited performance. In this work, we present a comprehensive framework, which enables the efficient training of deep TTFS-coded SNNs by employing backpropagation throuh time (BPTT) algorithm. We name the generalized TTFS coding method in our framework as latency coding. The framework includes: (1) a latency encoding (LE) module with feature extraction and straight-through estimators to address severe information loss in direct intensity-to-latency mapping and ensure smooth gradient flow; (2) relaxation of the strict single-spike constraint of traditional TTFS, allowing neurons of intermediate layers to fire multiple times to mitigating gradient vanishing in deep networks; (3) a temporal adaptive decision (TAD) loss function that dynamically weights supervision signals based on sample-dependent confidence, resolving the incompatibility between latency coding and standard cross-entropy loss. Experimental results demonstrate that our method achieves state-of-the-art accuracy in comparison to existing TTFS-coded SNNs with ultra-low inference latency and superior energy efficiency. The framework also demonstrates improved robustness against input corruptions. Our study investigates the characteristics and potential of latency coding in scenarios demanding rapid response, providing valuable insights for further exploiting the temporal learning capabilities of SNNs.
Paper Structure (25 sections, 16 equations, 6 figures, 5 tables)

This paper contains 25 sections, 16 equations, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background and Related Works
  3. Spiking Neuron Model
  4. Spiking Neuron
  5. Iterative LIF Model
  6. Training methods of SNNs
  7. TTFS Coding
  8. Methodology
  9. Latency Encoding Module
  10. Decoding with Membrane Potentials
  11. Temporal Adaptive Decision
  12. Experiments
  13. Comparison to Related Works
  14. Experimental Setup
  15. CIFAR-10 & CIFAR-100
  16. ...and 10 more sections

Figures (6)

  • Figure 1: Overall Framework
  • Figure 2: Performance evolution across severity levels on CIFAR-100-C. (a) Comparison of mCE from Severity 1 to 5. (b)-(f) Radar charts illustrating the error rates for 15 specific corruption types at each severity. Smaller polygon area indicates better robustness.
  • Figure 3: Temporal Similarity Matrix. Calculated on CIFAR100 test set. (a)(b) Rate-coded SNN. (c)(d) Latency-coded SNN.
  • Figure 4: Ablation study of the proposed LE module on (a) CIFAR-10 and (b) CIFAR-100.
  • Figure 5: Distribution of first spike time against sample difficulty. (a) Training with TAD loss. (b) Training with vanilla loss.
  • ...and 1 more figures