Obtaining Optimal Spiking Neural Network in Sequence Learning via CRNN-SNN Conversion

TL;DR

The paper tackles the difficulty of achieving ANN-level accuracy in sequence learning with SNNs by presenting a lossless CRNN-SNN conversion framework. It introduces the RBIF neuron to handle recurrent structures and two end-to-end pipelines, CNN-Morph and RNN-Morph, enabling lossless conversion from quantized CRNNs to SNNs with s-analog input encoding. Theoretical equivalence is established for both CNN and RNN branches, and extensive experiments on S-MNIST, PS-MNIST, and a collision-avoidance task show state-of-the-art performance and minimal conversion error. The work significantly advances the practical deployment of SNNs in sequential tasks by enabling ANN-level accuracy with low latency and energy-efficient neuromorphic implementations.

Abstract

Spiking neural networks (SNNs) are becoming a promising alternative to conventional artificial neural networks (ANNs) due to their rich neural dynamics and the implementation of energy-efficient neuromorphic chips. However, the non-differential binary communication mechanism makes SNN hard to converge to an ANN-level accuracy. When SNN encounters sequence learning, the situation becomes worse due to the difficulties in modeling long-range dependencies. To overcome these difficulties, researchers developed variants of LIF neurons and different surrogate gradients but still failed to obtain good results when the sequence became longer (e.g., $>$500). Unlike them, we obtain an optimal SNN in sequence learning by directly mapping parameters from a quantized CRNN. We design two sub-pipelines to support the end-to-end conversion of different structures in neural networks, which is called CNN-Morph (CNN $\rightarrow$ QCNN $\rightarrow$ BIFSNN) and RNN-Morph (RNN $\rightarrow$ QRNN $\rightarrow$ RBIFSNN). Using conversion pipelines and the s-analog encoding method, the conversion error of our framework is zero. Furthermore, we give the theoretical and experimental demonstration of the lossless CRNN-SNN conversion. Our results show the effectiveness of our method over short and long timescales tasks compared with the state-of-the-art learning- and conversion-based methods. We reach the highest accuracy of 99.16% (0.46 $\uparrow$) on S-MNIST, 94.95% (3.95 $\uparrow$) on PS-MNIST (sequence length of 784) respectively, and the lowest loss of 0.057 (0.013 $\downarrow$) within 8 time-steps in collision avoidance dataset.

Figures (5)

• Figure 1: RNN-RBIF conversion. A quantized RNN (left) is converted to its corresponding RBIFSNN (right) via QCRC framework without accuracy loss.
• Figure 2: The computational graphs of BIF neuron and the RBIF neuron. The recurrent connections in figure (b) are in spike form, which charges $s_{k-1}(t)$ to the H(t) at time-step t.
• Figure 3: Conversion pipelines. Conversion pipeline has two branches, where the top one is RNN-Morph and the bottom one is CNN-Morph. In general, both of the sub-pipelines can be divided into two steps: quantization and Neuron-Morph.
• Figure 4: Conversion error study. (left) The L1 Norm between the feature map of accumulated spiking output and the feature map of quantized activation output. (right) The sum output of the activation layers in QCRC. "C/L" refers to convolutional/linear layers and "R" denotes recurrent layers.
• Figure 5: The absolute distance between quantized ANN and SNN for two analog encoding methods. The results show the first 15 time steps in the sequence (left) and the entire sequence (right).

Theorems & Definitions (6)

• theorem thmcountertheorem
• proof
• theorem thmcountertheorem
• proof
• theorem thmcountertheorem
• proof