Zero-Shot Temporal Resolution Domain Adaptation for Spiking Neural Networks

TL;DR

Three novel domain adaptation methods for adapting neuron parameters to account for the change in time resolution without re-training on target time resolution are proposed and provide an alternative to-and in most cases significantly outperform-the existing reference method that consists of scaling only the time constant.

Abstract

Spiking Neural Networks (SNNs) are biologically-inspired deep neural networks that efficiently extract temporal information while offering promising gains in terms of energy efficiency and latency when deployed on neuromorphic devices. SNN parameters are sensitive to temporal resolution, leading to significant performance drops when the temporal resolution of target data during deployment is not the same as that of the source data used for training, especially when fine-tuning with the target data is not possible during deployment. To address this challenge, we propose three novel domain adaptation methods for adapting neuron parameters to account for the change in time resolution without re-training on target time resolution. The proposed methods are based on a mapping between neuron dynamics in SNNs and State Space Models (SSMs) and are applicable to general neuron models. We evaluate the proposed methods under spatio-temporal data tasks, namely the audio keyword spotting datasets SHD and MSWC, and the neuromorphic image NMINST dataset. Our methods provide an alternative to-and in most cases significantly outperform-the existing reference method that consists of scaling only the time constant. Notably, when the temporal resolution of the target data is double that of the source data, applying one of our proposed methods instead of the benchmark achieves classification accuracy of 89.5% instead of 53.0% on SHD, 93.6% instead of 38.8% on MSWC and 98.5% instead of 97.2% aon NMNIST. Moreover, our results show that high accuracy on high temporal resolution data can be obtained by time-efficient training on lower temporal resolution data.

Figures (7)

• Figure 1: Overview of the set-up. We investigate how models trained on data with a given source temporal resolution can be adapted for data with a different target temporal resolution, both in the Fine-to-Coarse and Coarse-to-Fine deployment directions.
• Figure 2: Feedforward SNN model architecture consisting of four layers, each with arbitrary number of neurons. An example neuron dynamics given as a zoom-in on a neuron.
• Figure 3: Membrane potential dynamics of a single adLIF neuron over time for different adaptation methods and time resolutions where $b_S$ (red circular) is the reference dynamics.
• Figure 4: Both plots show training + adaptation (wall-clock) time vs accuracy for the MSWC dataset and adLIF neuron, where the left plot uses fixed 50 epochs pre-training (results in Table \ref{['tab:sub:L2H_MSWC_Dataset']}), while the right plot uses early stopping pre-training. The gray dashed line represent the baseline performance of pre-training the model on the target resolution. Note that lines for the Integral and Expectation adaptation methods are overlapping.
• Figure 5: Example of voltage dynamics over time in a pre-trained network on $b_S=1$. Figure shows voltage dynamics of Fine-to-Coarse $b_S=1$ to $b_T=2$ experiment for different adaptation methods and time resolutions.

Theorems & Definitions (9)

• Remark 1
• Remark 2
• Remark 3
• Proposition 1
• Proposition 2
• Proposition 3
• Remark 4
• Remark 5
• Remark 6