Reservoir Computers with Random and Optimized Time-Shifts

TL;DR

This paper addresses enhancing reservoir computing by introducing time shifts to reservoir readouts. It presents a nonlinear RC model with ridge-regressed readouts and shows that random time shifts break readout synchronization, substantially reducing training and testing errors across chaotic tasks; it then introduces a simple, scalable optimization based on a first-order Taylor expansion to further reduce errors. Across Lorenz96, Lorenz, and Hindmarsh-Rose systems, random shifts yield large improvements, and optimized shifts often outperform random shifts, especially for challenging tasks and certain parameter regimes. The results highlight time-shift engineering as a practical hyperparameter tool for RCs and hint at a connection to Taken’s embedding concepts, suggesting avenues for principled design of delayed observations in dynamical systems.

Abstract

We investigate the effects of application of random time-shifts to the readouts of a reservoir computer in terms of both accuracy (training error) and performance (testing error.) For different choices of the reservoir parameters and different `tasks', we observe a substantial improvement in both accuracy and performance. We then develop a simple but effective technique to optimize the choice of the time-shifts, which we successfully test in numerical experiments.

Figures (7)

• Figure 1: Lorenz96 attractor. (A) Plot of the training error $\Delta_{tr}$ vs $\gamma$. The value of $\gamma$ that minimizes the training error is approximately equal to $0.9$. (B) Testing error vs $\gamma$, with a trend similar to that seen in (A). (C) Memory Capacity vs $\epsilon$ with $\gamma$ set equal to $0.9$. We find that the value of $\epsilon$ that maximizes the memory capacity is approximately equal to $0.8$.
• Figure 2: Lorenz96 system. The training error (A) and testing error (B) vs $\alpha$. The parameter $\alpha$ controls the interval over which the random time-shifts are taken $[0,\bar{\tau}\alpha]$, where $\bar{\tau}$ is the characteristic time-scale of the Lorenz task and was found to be 0.19. Error bars indicate the standard deviation over 50 iterations where each iteration corresponds to a different selection of the random time-shifts $\tau_i$, $\gamma=0.9$, $\epsilon=0.8$.
• Figure 3: Lorenz attractor. The training error (A) and testing error (B) vs $\alpha$. The parameter $\alpha$ controls the interval over which the random time-shifts are taken $[0,\bar{\tau}\alpha]$, where $\bar{\tau}$ is the characteristic time-scale of the Lorenz task and was found to be 0.3. Error bars indicate the standard deviation over 50 iterations where each iteration corresponds to a different selection of the random time-shifts $\tau_i$, $\gamma=1.3$, $\epsilon=2$.
• Figure 4: Hindmarsh-Rose attractor. The training error (A) and testing error (B) vs $\alpha$. The parameter $\alpha$ controls the interval over which the random time-shifts are taken $[0,\bar{\tau}\alpha]$, where $\bar{\tau}$ is the characteristic time-scale of the Lorenz task and was found to be 0.46. Error bars indicate the standard deviation over 50 iterations where each iteration corresponds to a different selection of the random time-shifts $\tau_i$, $\gamma=1.65$, $\epsilon=1$.
• Figure 5: Lorenz96 system. The training error (A) and testing error (B) vs $\gamma$ for both the cases of: randomly drawn time shifts and optimized time shifts. Here $\epsilon = 0.8$ and $\alpha = 4$.