Reservoir Computing with Generalized Readout based on Generalized Synchronization

TL;DR

The paper tackles the limitation of conventional reservoir computing's linear readout by introducing a generalized readout inspired by generalized synchronization. It formulates a generalized RC with a nonlinear readout that can be learned linearly, enabling higher-order functional approximations. Numerical experiments on Lorenz chaos show that quadratic and cubic readouts improve short- and long-term predictions and robustly reconstruct attractor dynamics. These results suggest a robust, scalable path for physically implementable RC architectures and invite further theoretical and practical exploration.

Abstract

Reservoir computing is a machine learning framework that exploits nonlinear dynamics, exhibiting significant computational capabilities. One of the defining characteristics of reservoir computing is its low cost and straightforward training algorithm, i.e. only the readout, given by a linear combination of reservoir variables, is trained. Inspired by recent mathematical studies based on dynamical system theory, in particular generalized synchronization, we propose a novel reservoir computing framework with generalized readout, including a nonlinear combination of reservoir variables. The first crucial advantage of using the generalized readout is its mathematical basis for improving information processing capabilities. Secondly, it is still within a linear learning framework, which preserves the original strength of reservoir computing. In summary, the generalized readout is naturally derived from mathematical theory and allows the extraction of useful basis functions from reservoir dynamics without sacrificing simplicity. In a numerical study, we find that introducing the generalized readout leads to a significant improvement in accuracy and an unexpected enhancement in robustness for the short- and long-term prediction of Lorenz chaos, with a particular focus on how to harness low-dimensional reservoir dynamics. A novel way and its advantages for physical implementations of reservoir computing with generalized readout are briefly discussed.

Figures (6)

• Figure 1: An illustration of the target and reservoir dynamics in phase space. As an example of the target dynamical system, the Rössler attractor is shown in the left panel. The right panel shows the projection of the reservoir dynamics driven by the Rössler dynamics onto the subspace spanned by the first three variables, i.e. ${\bf r}_{1},~{\bf r}_{2},~{\bf r}_{3}$. The red arrow depicts the schematic of the generalized synchronization map ${\bf f}$.
• Figure 2: Short-term prediction (open-loop).a, The top and bottom panels show the results using ${\mathcal{L}}$- and ${\mathcal{Q}}$-ESN, respectively. The left and right panels show the time series of the target (grey dashed) and prediction (red solid) and the phase space structures of the orbits. The colors represent the local error of the prediction, $\| {\bf y}_{t} - \hat{\bf y}_{t} \|$. b, The root mean square error (RMSE), $\sqrt{\langle \| {\bf y}_{t} - \hat{\bf y}_{t} \|^{2} \rangle_{T}}$, over the size of the network $N$. The red and blue dots show the RMSE using ${\mathcal{L}}$- and ${\mathcal{Q}}$-ESN, respectively.
• Figure 3: Long-term prediction (closed-loop) using the automated ${\mathcal{L}}$-ESN.a, The time series of the target (grey dashed) and prediction (red solid). The difference between the left and right panels lies in the realizations of the random numbers used for $A$ and $B$. b, The phase space structures of the orbits generated by the automated ${\mathcal{L}}$-ESN. The panels (a) -- (j) are results corresponding to the ten times realizations of the random numbers used for $A$ and $B$. The colors represent the local conjugacy error, ${\mathcal{E}}^{c}_{t}$. The length of the orbits shown is $T=50$.
• Figure 4: Long-term prediction (closed-loop) using the automated ${\mathcal{Q}}$-ESN. The same as Fig. 3, but the ${\mathcal{Q}}$-ESN is used instead of the ${\mathcal{L}}$-ESN.
• Figure 5: Probability density functions (PDF) of the variable $x$. The dashed lines show the PDF of the target Lorenz system $p(x)$. The red solid lines show the PDF $q(x)$ calculated from data generated by a, ${\mathcal{L}}$-ESN and b, ${\mathcal{Q}}$-ESN. Two panels of a and b correspond to the cases shown in Fig. 3 a and Fig. 4 a, respectively.