Synaptic Modulation using Interspike Intervals Increases Energy Efficiency of Spiking Neural Networks

TL;DR

This paper introduces ISI Modulated SNNs (IMSNNs), a novel energy-efficient Spiking Neural Network that uses an ISI-dependent Gaussian synapse to modulate spike impact. By backpropagating through time and leveraging an ISI gradient, the learning algorithm selectively promotes higher ISIs, thereby reducing spike activity without sacrificing classification accuracy. Empirical results on MNIST and FashionMNIST show IMSNNs can reduce spike counts by up to 90% with comparable performance, including spiking convolutional networks that achieve substantial spike reductions. This approach directly targets spike-based energy consumption in SNNs and offers a gradient-based alternative to traditional pruning and quantization aimed at energy efficiency.

Abstract

Despite basic differences between Spiking Neural Networks (SNN) and Artificial Neural Networks (ANN), most research on SNNs involve adapting ANN-based methods for SNNs. Pruning (dropping connections) and quantization (reducing precision) are often used to improve energy efficiency of SNNs. These methods are very effective for ANNs whose energy needs are determined by signals transmitted on synapses. However, the event-driven paradigm in SNNs implies that energy is consumed by spikes. In this paper, we propose a new synapse model whose weights are modulated by Interspike Intervals (ISI) i.e. time difference between two spikes. SNNs composed of this synapse model, termed ISI Modulated SNNs (IMSNN), can use gradient descent to estimate how the ISI of a neuron changes after updating its synaptic parameters. A higher ISI implies fewer spikes and vice-versa. The learning algorithm for IMSNNs exploits this information to selectively propagate gradients such that learning is achieved by increasing the ISIs resulting in a network that generates fewer spikes. The performance of IMSNNs with dense and convolutional layers have been evaluated in terms of classification accuracy and the number of spikes using the MNIST and FashionMNIST datasets. The performance comparison with conventional SNNs shows that IMSNNs exhibit upto 90% reduction in the number of spikes while maintaining similar classification accuracy.

Figures (3)

• Figure 1: In IMSNNs, the contribution of a spike to the postsynaptic potential depends on the Interspike Interval (ISI) of the presynaptic neurons. a), b) and c) show the postsynaptic potential for three different scenarios, namely $ISI < \mu$, $ISI = \mu$ and $ISI > \mu$, respectively. The presynaptic pattern in each case has 4 spikes but the postsynaptic potential contributed by these spikes is different across the three scenarios. The synaptic parameter $\mu$ represents the ISI that result in maximum postsynaptic potential (see b)). As ISI deviates from $\mu$, the contribution to postsynaptic potential goes down (see a) and c)).
• Figure 2: Proposed synapse model used in IMSNNs. $\mu^{(l)}_{ij}, \sigma^{(l)}_{ij}$ and $w^{(l)}_{ij}$ represent the mean, width and height of the synapse, respectively. Note that only heights are learnt in this paper. Mean and width are assigned random values at the start of training and are not learnt during training.
• Figure 3: Output of LIF neurons with a single input synapse in three networks. First network has a conventional synapse with fixed weight throughout the simulation. Second and third networks use the proposed synapse with $\mu$ set to 10ms and 15 ms, respectively.