Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness

TL;DR

This work introduces Mixed Time-step Training (MTT), a novel method that improves the temporal flexibility of SNNs, making SNNs adaptive to diverse temporal structures, and reports the first work to report the results of large-scale SNN deployment on fully event-driven scenarios.

Abstract

Spiking Neural Networks (SNNs), models inspired by neural mechanisms in the brain, allow for energy-efficient implementation on neuromorphic hardware. However, SNNs trained with current direct training approaches are constrained to a specific time step. This "temporal inflexibility" 1) hinders SNNs' deployment on time-step-free fully event-driven chips and 2) prevents energy-performance balance based on dynamic inference time steps. In this study, we first explore the feasibility of training SNNs that generalize across different time steps. We then introduce Mixed Time-step Training (MTT), a novel method that improves the temporal flexibility of SNNs, making SNNs adaptive to diverse temporal structures. During each iteration of MTT, random time steps are assigned to different SNN stages, with spikes transmitted between stages via communication modules. After training, the weights are deployed and evaluated on both time-stepped and fully event-driven platforms. Experimental results show that models trained by MTT gain remarkable temporal flexibility, friendliness for both event-driven and clock-driven deployment (nearly lossless on N-MNIST and 10.1% higher than standard methods on CIFAR10-DVS), enhanced network generalization, and near SOTA performance. To the best of our knowledge, this is the first work to report the results of large-scale SNN deployment on fully event-driven scenarios.

Figures (15)

• Figure 1: The workflow of MTT pipeline. We first partition SNN into $\rm G$ stages. In each iteration, we sample $s$ temporal configs ${\bm t}^{(1)}, ..., {\bm t}^{(s)}$, each assigning a set of random time steps to different stages (for $j$-th sampled config, $T_i = t_i^{(j)}$). These configurations create $s$ partitioned SNNs with distinct temporal structures, all sharing the same weights. To update the shared weights, we backpropagate the sum of the $s$ losses to obtain the gradient. Models trained with MTT exhibit temporal flexibility, which leads to their adaptation to any time step and friendliness with fully event-driven chips.
• Figure 1: Inference accuracy of ResNet18 on CIFAR100 by naive mixture training vs. standard direct training. "SDT*": SNNs independently trained with SDT at each T. "SDT": single SNN trained at T=6 and infers at other T.
• Figure 2: SNN partitioned for Mixed Timestep Training.
• Figure 2: Accuracy of different inference time steps.
• Figure 3: (A) Downsampling TTM when $t_{in}$=5 and $t_{out}$=3. (B) Upsampling TTM when $t_{in}$=3 and $t_{out}$=5.