Time Shifts to Reduce the Size of Reservoir Computers

TL;DR

The paper tackles the challenge of scaling reservoir computers by separating a small nonlinear reservoir from a large, time-shifted output matrix. It shows that increasing the covariance rank ${Γ = \text{rank}(Ω_1^T Ω_1)}$ and the memory capacity ${MC}$ via time-shifted signals can dramatically improve predictive accuracy without proportionally increasing the number of nonlinear nodes. The authors validate the approach across tanh, polyODE, and opto-electronic delay-based RCs and demonstrate a concrete experimental gain in an opto-electronic delay RC, achieving high performance with far fewer physical nodes. The work suggests a practical route to faster, more power-efficient analog RC hardware by leveraging time-shifted outputs to boost effective dimensionality and memory.

Abstract

A reservoir computer is a type of dynamical system arranged to do computation. Typically, a reservoir computer is constructed by connecting a large number of nonlinear nodes in a network that includes recurrent connections. In order to achieve accurate results, the reservoir usually contains hundreds to thousands of nodes. This high dimensionality makes it difficult to analyze the reservoir computer using tools from dynamical systems theory. Additionally, the need to create and connect large numbers of nonlinear nodes makes it difficult to design and build analog reservoir computers that can be faster and consume less power than digital reservoir computers. We demonstrate here that a reservoir computer may be divided into two parts; a small set of nonlinear nodes (the reservoir), and a separate set of time-shifted reservoir output signals. The time-shifted output signals serve to increase the rank and memory of the reservoir computer, and the set of nonlinear nodes may create an embedding of the input dynamical system. We use this time-shifting technique to obtain excellent performance from an opto-electronic delay-based reservoir computer with only a small number of virtual nodes. Because only a few nonlinear nodes are required, construction of a reservoir computer becomes much easier, and delay-based reservoir computers can operate at much higher speeds.

Figures (6)

• Figure 1: Mean testing error $\left <\Delta_{tx} \right >$ as a function of covariance rank $\Gamma$ for Lorenz or Rössler $x$ signals driving either a tanh, a polynomial ODE or a opto-electronic reservoir computer. The reservoir computers were trained on the $z$ signals from the corresponding input systems. Reservoir computers containing 50 or 100 nodes were used. The tanh data has twice the rank because the set of reservoir computer signals $r_i(t)$ were supplemented with their squares $r_i^2(t)$.
• Figure 2: Mean testing error $\left <\Delta_{tx} \right >$ as a function of memory capacity MC for Lorenz or Rössler $x$ signals driving either a tanh, a polynomial ODE or a opto-electronic reservoir computer. The reservoir computers were trained on the $z$ signals from the corresponding input systems. Reservoir computers containing 50 or 100 nodes were used.
• Figure 3: The top plot is the mean testing error for the opto-electronic reservoir computer when the input signal is the Lorenz $x$ signal and the training signal is the $z$ signal. The trace labeled $\Omega_1$ is the error for the original reservoir computer with $M_1$ nodes. The traces labeled as $\Omega_2$ are for the time-shifted matrix with different fixed values of $M_2$. The middle plot shows the covariance rank $\Gamma$ while the bottom plot shows the memory capacity MC.
• Figure 4: The top plot is the mean training error for the opto-electronic reservoir computer when the input signal is the Rössler $x$ signal and the training signal is the $z$ signal. The trace labeled $\Omega_1$ is the error for the original reservoir computer with $M_1$ nodes. The traces labeled as $\Omega_2$ are for the time-shifted matrix with different fixed values of $M_2$. The bottom plot shows the covariance rank $\Gamma$.
• Figure 5: Mean testing errors $\left< \Delta_{tx} \right>$ as a function of the number of nodes $M_1$ in the non-time-shifted reservoir computer. The red symbols are for the reservoir computer without time shifts, while the blue symbols are for fits using the time-shifted matrix $\Omega_2$ with 200 nodes. These plots follow the same trends as the simulations in Figs. \ref{['laserlorshifterr']} and \ref{['laserrossshifterr']}.