A Hybrid Neural Coding Approach for Pattern Recognition with Spiking Neural Networks

TL;DR

The paper presents a hybrid neural coding framework for spiking neural networks that integrates diverse coding schemes—direct, burst, phase, and TTFS—across input, hidden, and output layers. A neural coding zoo and a layer-wise learning strategy (LTL) enable practical construction and training of hybrid coding SNNs, demonstrated on image classification and sound localization. Empirical results show competitive accuracy with substantially reduced inference latency and energy consumption, alongside strong noise robustness, compared to homogeneous-coding SNNs. The approach advances neuromorphic computing by enabling task-specific, energy-efficient pattern recognition through holistic coding design and scalable layer-wise optimization.

Abstract

Recently, brain-inspired spiking neural networks (SNNs) have demonstrated promising capabilities in solving pattern recognition tasks. However, these SNNs are grounded on homogeneous neurons that utilize a uniform neural coding for information representation. Given that each neural coding scheme possesses its own merits and drawbacks, these SNNs encounter challenges in achieving optimal performance such as accuracy, response time, efficiency, and robustness, all of which are crucial for practical applications. In this study, we argue that SNN architectures should be holistically designed to incorporate heterogeneous coding schemes. As an initial exploration in this direction, we propose a hybrid neural coding and learning framework, which encompasses a neural coding zoo with diverse neural coding schemes discovered in neuroscience. Additionally, it incorporates a flexible neural coding assignment strategy to accommodate task-specific requirements, along with novel layer-wise learning methods to effectively implement hybrid coding SNNs. We demonstrate the superiority of the proposed framework on image classification and sound localization tasks. Specifically, the proposed hybrid coding SNNs achieve comparable accuracy to state-of-the-art SNNs, while exhibiting significantly reduced inference latency and energy consumption, as well as high noise robustness. This study yields valuable insights into hybrid neural coding designs, paving the way for developing high-performance neuromorphic systems.

Figures (9)

• Figure 1: Illustration of the proposed hybrid neural coding and learning framework in solving the image classification task. The proposed framework consists of three components: Neural Coding Zoo (upper box), Assignment Strategy (gray dotted line), and Layer-wise Learning Method (red dotted line). First of all, the neural coding zoo is a comprehensive collection of the most representative neural coding schemes discovered in neuroscience. According to the specific task requirements of image classification, the assignment strategy has been further designed to assign heterogeneous neural coding schemes from the neural coding zoo to different network layers: direct coding for high-fidelity input representation, burst coding for reliable and rapid hidden feature representation, and TTFS coding to ensure fast and efficient output decision-making. Finally, the layer-wise learning methods are introduced to independently train the hidden and output layers, so as to achieve the desired coding schemes. Here, $\boldsymbol{y}^L$ represents the outputs of the hybrid coding SNN, $\boldsymbol{y}^{label}$ denotes the one-hot labels, $\hat{\boldsymbol{y}}^l$ signifies the activation values of neurons at hidden layer $l$ in the teacher ANN.
• Figure 2: Comparison of computational performance and biological plausibility among diverse candidate neural coding schemes.
• Figure 3: Illustration of the phase encoding front-end used for the sound localization task, where two microphones are selected for demonstration. (a) The raw audio is decomposed into $n$ single-tone sinusoidal signals by fast Fourier transform (FFT). (b) The first peak time of each sinusoidal signal is encoded into a precise-timing spike, which represents the arrival time of each sound. (c)$N_{\tau}$ coincidence detection neurons are employed to detect the phase difference between two microphones. If a coincidence detection neuron detects its predefined phase difference, it emits a spike. (d) To reduce the computational cost of post-processing, the encoded spike trains of the $n$ single tones are further grouped into $N_c$ channels via constant Q transform (CQT).
• Figure 4: Test accuracy as a function of inference time on the CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets with (a,b,c) VGG-16 and (d,e,f) ResNet-20 architectures. Note that for the proposed hybrid coding approach, we only report the average inference time as the decision is made as soon as the first output spike is generated.
• Figure 5: Decision time distribution for the proposed hybrid coding models (D+B+T and D+R+T). The decision times for each sample vary according to the TTFS coding mechanism. Green Bar: the proportion of correct decisions. Red Bar: the proportion of misclassified samples.