Gated Parametric Neuron for Spike-based Audio Recognition

TL;DR

The paper tackles the vanishing gradient problem and fixed neuronal parameters in leaky integrate-and-fire (LIF) neurons used for spike-based audio recognition. It proposes a gated parametric neuron (GPN) with four gates that regulate membrane leakage, inputs, firing thresholds, and an auxiliary bypass path, enabling both gradient flow and spatio-temporal heterogeneity without manual parameter initialization. Empirical results on SHD and SSC show GPN outperforms standard SNNs and many baselines, while ablations confirm the gates’ critical roles and gradient analyses demonstrate improved information flow over long sequences. The approach also reveals meaningful learned distributions of neuronal parameters, supporting the feasibility of hybrid RNN-SNN dynamics for sequence-rich tasks and highlighting GPN’s potential for neuromorphic audio processing.

Abstract

Spiking neural networks (SNNs) aim to simulate real neural networks in the human brain with biologically plausible neurons. The leaky integrate-and-fire (LIF) neuron is one of the most widely studied SNN architectures. However, it has the vanishing gradient problem when trained with backpropagation. Additionally, its neuronal parameters are often manually specified and fixed, in contrast to the heterogeneity of real neurons in the human brain. This paper proposes a gated parametric neuron (GPN) to process spatio-temporal information effectively with the gating mechanism. Compared with the LIF neuron, the GPN has two distinguishing advantages: 1) it copes well with the vanishing gradients by improving the flow of gradient propagation; and, 2) it learns spatio-temporal heterogeneous neuronal parameters automatically. Additionally, we use the same gate structure to eliminate initial neuronal parameter selection and design a hybrid recurrent neural network-SNN structure. Experiments on two spike-based audio datasets demonstrated that the GPN network outperformed several state-of-the-art SNNs, could mitigate vanishing gradients, and had spatio-temporal heterogeneous parameters. Our work shows the ability of SNNs to handle long-term dependencies and achieve high performance simultaneously.

Figures (9)

• Figure 2: The LIF neuron. (a) Structure of the LIF neuron. (b) The membrane potential of the LIF neuron rises with the current input. Without input, it decays back to the reset potential. When the membrane potential exceeds the threshold potential, the neuron fires and resets the membrane potential.
• Figure 3: The unrolled computational graph of the LIF network. The red dotted lines represent the detached backward pathways.
• Figure 4: Details of the proposed GPN and the structure of the GPN network. The upper left corner shows the internal structure of GPN, where the green circles represent the membrane potential states, the orange circles the input currents and output spikes, and the yellow boxes the four gate structures of GPN. The spatio-temporal structure of the GPN network is shown at the bottom.
• Figure 5: Structures with the bypass gate. (a) A LIF network with bypass gate; and, (b) a network similar to the vanilla RNN.
• Figure 6: The two spike-based audio datasets. (a) A sample from the SHD dataset; and, (b) a sample from the SSC dataset.