Balanced Resonate-and-Fire Neurons

TL;DR

It is shown that networks of BRF neurons achieve overall higher task performance, produce only a fraction of the spikes, and require significantly fewer parameters as compared to modern RSNNs, indicating that the BRF-RSNN is a strong candidate for future large-scale RSNN architectures, further lines of research in SNN methodology, and more efficient hardware implementations.

Abstract

The resonate-and-fire (RF) neuron, introduced over two decades ago, is a simple, efficient, yet biologically plausible spiking neuron model, which can extract frequency patterns within the time domain due to its resonating membrane dynamics. However, previous RF formulations suffer from intrinsic shortcomings that limit effective learning and prevent exploiting the principled advantage of RF neurons. Here, we introduce the balanced RF (BRF) neuron, which alleviates some of the intrinsic limitations of vanilla RF neurons and demonstrates its effectiveness within recurrent spiking neural networks (RSNNs) on various sequence learning tasks. We show that networks of BRF neurons achieve overall higher task performance, produce only a fraction of the spikes, and require significantly fewer parameters as compared to modern RSNNs. Moreover, BRF-RSNN consistently provide much faster and more stable training convergence, even when bridging many hundreds of time steps during backpropagation through time (BPTT). These results underscore that our BRF-RSNN is a strong candidate for future large-scale RSNN architectures, further lines of research in SNN methodology, and more efficient hardware implementations.

Figures (13)

• Figure 1: Membrane dynamics of two RF neurons. The four excitatory input spikes are in phase with the neuron's angular frequency $\omega = 10$. The inhibitory spike at half-phase enhances the sensitivity of the neuron to forthcoming excitatory input. Depending on its parameterization, the neuron exhibits divergence behavior (top) with $b = -0.3$ and $\delta = 0.01$ or convergence behavior (bottom) with $b = -1$. $I(t) \in \mathbb{R}$ and $u_1(t),u_2(t) \in \mathbb{C}$ refer to the injected current and the membrane potential of the neuron, respectively.
• Figure 2: Membrane potential $u(t)$ and spiking response $z(t)$ of an RF neuron without refractory period (RP) or smooth reset (SR) (orange) compared to both (blue) for the given input signal $I(t)$.
• Figure 3: RF neuron frequency response plots for exemplary omega $\omega$ and b-offset $b'$ combinations with $\delta = 0.001$.
• Figure 4: Overview of the datasets. Examplary MNIST image and its corresponding sequential and permuted representations. Common pixel row outlined in red on MNIST and S-MNIST sample. ECG sample after level cross encoding. SHD sample after preprocessing.
• Figure 5: Top row: S-MNIST, PS-MNIST, ECG, and SHD learning curve between BRF, RF and ALIF model. Each curve averaged per epoch (solid line) with standard deviation (shaded area) over five runs. The dot on the accuracy curves depict the point at which 95 % of the final accuracy was reached. Bottom row: S-MNIST, PS-MNIST, ECG, and SHD initial and optimized BRF parameter combinations: angular frequency $\omega$ and b-offset $b'$ for all runs. Dashed line is the divergence boundary. RF models are simulated without reset. For convergence with traditional reset see \ref{['sec:reset']}.