Hybridizing Traditional and Next-Generation Reservoir Computing to Accurately and Efficiently Forecast Dynamical Systems

TL;DR

This work tackles the challenge of forecasting chaotic dynamical systems under computational constraints by hybridizing traditional reservoir computers with next-generation reservoir computers. By concatenating the RC state with NGRC nonlinear features into a single hybrid representation and training a ridge-readout, the method preserves the robustness of RCs while leveraging the efficiency of NGRCs. The hybrid approach yields superior short-term forecasts and more faithful long-term climate statistics, particularly when both components are individually limited by small reservoir size or data sparsity. This yields substantial computational gains and robustness, making the method attractive for resource-constrained forecasting tasks across chaotic systems.

Abstract

Reservoir computers (RCs) are powerful machine learning architectures for time series prediction. Recently, next generation reservoir computers (NGRCs) have been introduced, offering distinct advantages over RCs, such as reduced computational expense and lower training data requirements. However, NGRCs have their own practical difficulties, including sensitivity to sampling time and type of nonlinearities in the data. Here, we introduce a hybrid RC-NGRC approach for time series forecasting of dynamical systems. We show that our hybrid approach can produce accurate short term predictions and capture the long term statistics of chaotic dynamical systems in situations where the RC and NGRC components alone are insufficient, e.g., due to constraints from limited computational resources, sub-optimal hyperparameters, sparsely-sampled training data, etc. Under these conditions, we show for multiple model chaotic systems that the hybrid RC-NGRC method with a small reservoir can achieve prediction performance approaching that of a traditional RC with a much larger reservoir, illustrating that the hybrid approach can offer significant gains in computational efficiency over traditional RCs while simultaneously addressing some of the limitations of NGRCs. Our results suggest that hybrid RC-NGRC approach may be particularly beneficial in cases when computational efficiency is a high priority and an NGRC alone is not adequate.

Figures (7)

• Figure 1: a) Reservoir computer (RC) schematic. Time series observations $\vb{u}(t)$ are fed into a high-dimensional reservoir with state $\vb{r}(t)$ via an input matrix $B$, then an output matrix $W$ is trained to predict the next data point in the series (i.e., at time $t + \tau$). Predictions at times $t > t_{\text{train}}$ are made by switching to autonomous mode in which outputs of the reservoir are repeatedly fed back in as input (dashed line). b) Next-generation reservoir computers (NGRCs) replace the reservoir with a nonlinear feature vector $\vb{O}(t)$ that is constructed using time-delayed observations. c) Our hybrid RC-NGRC prediction approach uses a hybrid feature vector $\vb{H}(t)$ that is the concatenation of a reservoir state with a NGRC feature vector in order to produce a prediction.
• Figure 2: a) Representative examples of RC, NGRC, and hybrid RC-NGRC autonomous predictions of the Lorenz system ($x$ component shown), where a small reservoir ($N=50$) and large time step ($\tau = 0.06$) are used, limiting RC and NGRC performance. Valid prediction time (VPT) indicated by the vertical dashed line. b) Distributions of VPTs for RC, NGRC, and RC-NGRC predictions, where each trial is done on new initial conditions using a new reservoir realization. The hybrid RC-NGRC shows substantially stronger short term predictive power than either the RC or NGRC alone. Horizontal lines: quartiles (100 trials).
• Figure 3: a) Representative examples of long-term phase space trajectories of the RC, NGRC, and hybrid RC-NGRC autonomous predictions (predictions extended from Figure \ref{['fig:hybrid_VPTs']}a)). Though RC and NGRC reconstruct the Lorenz attractor in some trials, only the hybrid RC-NGRC prediction reliably reconstructs the attractor of the true system across trials. b) Power spectra of $z$ component of autonomous predictions for the different methods. Only the hybrid RC-NGRC prediction reliably reproduces the spectrum of the Lorenz system.
• Figure 4: a) Mean valid prediction times for the Lorenz system versus number of nodes in the reservoir. Although RC performance is poor at small $N$, and NGRC performance is modest due to using a large timestep ($\tau = 0.06$), the hybrid RC-NGRC performs well throughout, specifically providing a substantial advantage over both RC and NGRC at small $N$. Note that the hybrid RC-NGRC approach with reservoir size N = 100 approximately matches that of a pure RC with N = 500. b) Mean valid prediction times for the Lorenz system versus time step size $\tau$ in the training data. As time step is adjusted, the number of training data points $n_{\text{train}}$ is kept constant. The hybrid RC-NGRC shows the greatest advantage in predictive power over the RC or NGRC alone when using a large time step. Reservoir size $N = 50$. Error bars and band: standard error of the mean (64 trials).
• Figure 5: The hybrid RC-NGRC approach exhibits reduced sensitivity to reservoir hyperparameters compared to RCs. Shown here: mean VPT vs. input matrix scaling $\sigma$ (other examples shown in Supplementary Materials). Error bars and band: standard error of the mean (64 trials).