Efficient and Effective Time-Series Forecasting with Spiking Neural Networks

TL;DR

This paper proposes a framework for SNNs in time-series forecasting tasks, leveraging the efficiency of spiking neurons in processing temporal information and demonstrates that its proposed SNN-based approaches achieve comparable or superior results to traditional time-series forecasting methods on diverse benchmarks with much less energy consumption.

Abstract

Spiking neural networks (SNNs), inspired by the spiking behavior of biological neurons, provide a unique pathway for capturing the intricacies of temporal data. However, applying SNNs to time-series forecasting is challenging due to difficulties in effective temporal alignment, complexities in encoding processes, and the absence of standardized guidelines for model selection. In this paper, we propose a framework for SNNs in time-series forecasting tasks, leveraging the efficiency of spiking neurons in processing temporal information. Through a series of experiments, we demonstrate that our proposed SNN-based approaches achieve comparable or superior results to traditional time-series forecasting methods on diverse benchmarks with much less energy consumption. Furthermore, we conduct detailed analysis experiments to assess the SNN's capacity to capture temporal dependencies within time-series data, offering valuable insights into its nuanced strengths and effectiveness in modeling the intricate dynamics of temporal data. Our study contributes to the expanding field of SNNs and offers a promising alternative for time-series forecasting tasks, presenting a pathway for the development of more biologically inspired and temporally aware forecasting models. Our code is available at https://github.com/microsoft/SeqSNN.

Figures (6)

• Figure 1: A recurrent representation of a leaky integrate-and-fire (LIF) neuron. The membrane potential $U(t-1)$ and spike $S(t-1)$ at time step $t-1$ are derived from their counterparts at time step $t-2$ and undergo processing to yield $U(t)$ and $S(t)$ at time step $t$.
• Figure 2: An overview of our framework for SNNs in time-series forecasting. Given an input time-series sample $\mathbf{X} = \{\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_T\}$ with $T$, our goal is to predict the values in the following $L$ time steps $\mathbf{Y} = \{\mathbf{x}_{T+1}, \mathbf{x}_{T+2}, \dots, \mathbf{x}_{T+L}\}$. Firstly, a spike encoder will be used to generate spike trains with $T_s$ spiking time steps from the original data every $\Delta t$ time step. After being encoded, time-series data will be converted to spike trains ($B \times T_s \times T \times C$) and will be fed into SNNs. We provide three SNNs: (a) Spike-TCN; (b) Spike-RNN; and (c) Spike-Transformer. Finally, the spike trains will be converted to floating-point values by a projection layer.
• Figure 3: Critical Difference (CD) diagram of all methods in Table \ref{['tab:main_result']} on time series forecasting tasks with a confidence level of $95\%$.
• Figure 4: The impact of two crucial hyper-parameters in SNNs: time Steps $T_s$ and the decay rate $\beta$. (a) and (b): R$^2$ versus $T_s$ on Metr-la and Solar respectively. (c) and (d): R$^2$ versus $\beta$ on Metr-la and Solar respectively. The horizon $L$ of these experiments is set to $24$.
• Figure 5: A prediction slice ($T=20, L=80$) of Spike-TCN, Spike-RNN, and iSpikformer on synthetic time-series data. (a) Prediction slice on low-frequency data. (a) Prediction slice on high-frequency data.