Neuromorphic on-chip reservoir computing with spiking neural network architectures

TL;DR

This study investigates the application of integrate-and-fire neurons within reservoir computing frameworks for two distinct tasks: capturing chaotic dynamics of the H\'enon map and forecasting the Mackey-Glass time series.

Abstract

Reservoir computing is a promising approach for harnessing the computational power of recurrent neural networks while dramatically simplifying training. This paper investigates the application of integrate-and-fire neurons within reservoir computing frameworks for two distinct tasks: capturing chaotic dynamics of the Hénon map and forecasting the Mackey-Glass time series. Integrate-and-fire neurons can be implemented in low-power neuromorphic architectures such as Intel Loihi. We explore the impact of network topologies created through random interactions on the reservoir's performance. Our study reveals task-specific variations in network effectiveness, highlighting the importance of tailored architectures for distinct computational tasks. To identify optimal network configurations, we employ a meta-learning approach combined with simulated annealing. This method efficiently explores the space of possible network structures, identifying architectures that excel in different scenarios. The resulting networks demonstrate a range of behaviors, showcasing how inherent architectural features influence task-specific capabilities. We study the reservoir computing performance using a custom integrate-and-fire code, Intel's Lava neuromorphic computing software framework, and via an on-chip implementation in Loihi. We conclude with an analysis of the energy performance of the Loihi architecture.

Figures (9)

• Figure 1: Attractor of the Hénon map, for $a = 1.4$, $b = 0.3$, starting from $x=y=0$.
• Figure 2: 2-D representation of results of Hénon map task with Erdős-Rényi graph network. This simulation included a network of $M=100$ neurons with a connection probability between any two neurons of $2/M$. The input was spatially encoded as described in the main paper. This random graph produced $NRMSE = 0.178$
• Figure 3: Left: An example of random small world. On the right are the results of the Hénon map tasks when the network. Every neuron in the small world network was connected to its immediate neighbors and additional connections were added with a probability of $2/M$, where $M = 100$. Input was encoded with $M_{in} = 50$ neurons, these input neurons occurred in every other neuron around the circle. This graph produced a poor $NRMSE$ of $0.204$
• Figure 4: Left: The hand-picked network, equivalent to a ring lattice with an additional external ring of input neurons. Right: Reconstructed attractor of the Hénon map with the hand-picked network. This network consisted of $M = 100$ total neurons, $M_{in} = 50$ of them being the input neurons, and the remaining 50 making a ring connection with each neuron in the ring sending spikes to their immediate neighbors. The hand-picked network achieved $NRMSE = 0.0505$
• Figure 5: Top left (TL). NRMSE vs Iteration of meta-learning algorithm for the Hénon Map Task. Top right (TR) and Bottom Left (BL). Two-dimensional representation of Hénon Map prediction task results at the start of the meta-learning algorithm in (TL) and at the end of $1000$ steps in TR.BR. Membrane Voltage vs Time for each neuron in the network from time $[0,3]$.