Spiking Neural Networks with Random Network Architecture

TL;DR

This work tackles the challenge of training spiking neural networks by introducing RanSNN, a framework with fixed random hidden layers and a trainable final readout. Inputs are encoded as Poisson spike trains and processed by Leaky Integrate-and-Fire neurons, with only the last layer learned via linear optimization, enabling efficient training. Empirical results on MNIST-family benchmarks show competitive accuracy with substantially reduced training time compared to surrogate-gradient methods, and the authors provide extensive analyses of hyper-parameters and random-weight effects. The approach offers a scalable, energy-efficient alternative for certain SNN applications, while acknowledging limitations in generalizing to networks requiring learned convolutional kernels and more complex decoding.

Abstract

The spiking neural network, known as the third generation neural network, is an important network paradigm. Due to its mode of information propagation that follows biological rationality, the spiking neural network has strong energy efficiency and has advantages in complex high-energy application scenarios. However, unlike the artificial neural network (ANN) which has a mature and unified framework, the SNN models and training methods have not yet been widely unified due to the discontinuous and non-differentiable property of the firing mechanism. Although several algorithms for training spiking neural networks have been proposed in the subsequent development process, some fundamental issues remain unsolved. Inspired by random network design, this work proposes a new architecture for spiking neural networks, RanSNN, where only part of the network weights need training and all the classic training methods can be adopted. Compared with traditional training methods for spiking neural networks, it greatly improves the training efficiency while ensuring the training performance, and also has good versatility and stability as validated by benchmark tests.

Figures (6)

• Figure 1: The composition of the SNN and the process of information propagation are as follows. The yellow neurons represent the Leaky Integrate-and-Fire (LIF) neuron model. The green connection weights are randomly generated and remain fixed (represented as green solid lines). In the last layer, the spike signals are accumulated and then undergo a linear mapping as the output (represented as blue dashed lines). The network task is completed by training this linear mapping
• Figure 2: Performance of SG and RanSNN: the convergence of train accuracy, test accuracy, and loss as iteration increases. (a)(c)(e) for RanSNN and (b)(d)(f) for SG respectively.
• Figure 3: The impact of hyper-parameters on the results. (a)The leakage parameter $\beta$; (b)The number of neurons in the hidden layer; (c)The length of the spike train time.
• Figure 4: Performance under different random weights generation by uniform distributions:(a) ${\rm U}(-a,a)$; (b)${\rm U}(a,a+0.1)$
• Figure 5: Performance under different random weights generation by normal distributions : (a) normal distribution ${\rm N}(a,1)$; (b) normal distribution ${\rm N}(0,a)$.