Improving the prediction of spatio-temporal chaos by combining parallel reservoir computing with dimensionality reduction

TL;DR

The paper tackles the challenge of predicting high-dimensional spatio-temporal chaos by combining parallel reservoir computing with latent-space dimensionality reduction. It demonstrates that running multiple small reservoirs in parallel, optionally augmented with latent-space transformations such as PCA or FFT, can outperform a single large reservoir while reducing computational cost. The key finding is that the joint use of parallel reservoirs and latent-space predictions yields substantial gains for small reservoirs, and this approach is robust to training data noise, with clear guidelines for hyperparameters. Collectively, the method offers a scalable path to accurate long-horizon predictions of chaotic spatio-temporal dynamics, demonstrated on the Kuramoto–Sivashinsky equation.

Abstract

Reservoir computers can be used to predict time series generated by spatio-temporal chaotic systems. Using multiple reservoirs in parallel has shown improved performances for these predictions, by effectively reducing the input dimensionality of each reservoir. Similarly, one may further reduce the dimensionality of the input data by transforming to a lower-dimensional latent space. Combining both approaches, we show that using dimensionality-reduced latent space predictions for parallel reservoir computing not only reduces computational costs, but also leads to better prediction results for small to medium reservoir sizes. In the combined approach we further demonstrate that dimensionality reduction improves small-reservoir predictions regardless of noise contaminating the training data. The benefit of dimensionality-reduced parallel reservoir computing is illustrated and evaluated on the basis of the prediction of the one-dimensional Kuramoto-Sivashinsky equation.

Figures (13)

• Figure 1: Time series and iterative prediction of the Kuramoto--Sivashinsky model. Panel a shows the temporal evolution of the trajectory following Eq. \ref{['eq:ks']}, i. e. ground truth data. In Panel b, the iterative prediction of the time series using the combined approach of parallel reservoirs with dimensionality reduction (see Sec. \ref{['chap:ext_lat_parres']}) is shown. Panel c shows the difference between the ground truth and the prediction. The valid time of the prediction $t_{\mathrm{val}} \approx 10$ Lyapunov times is marked by the dashed black line in all panels.
• Figure 2: Modifications of a single time series prediction step to enhance performance of reservoir computing.a The parallel reservoir approach is shown for $M=2$ parallel reservoirs. The input domain is divided into two subdomains, each predicted by its own reservoir. Note that the input domains share overlapping neighbourhoods, while the prediction domains are disjoint. b Dimensionality-reduced latent space predictions are shown using the PCA as linear transformation $\mathcal{L}$ of the system state. In a second step, only the largest $\eta=75\%$ of the PCA components are used as reservoir input. While the reservoir's input is only a portion of the transformed data, all transformed system variables are predicted. The inverse transformation $\mathcal{L}^{-1}$ maps the state back to the original space.
• Figure 3: Parallel reservoirs improve prediction performance for fixed node numbers. The mean valid time is shown for varying reservoir sizes $N$ (logarithmic scale on the $x$-axis) and parallel reservoirs $M$ (coloured lines). Optimal hyperparameters are determined for each case individually and mean performance is averaged over 500 predictions, obtained from 10 random reservoir realisations, each evaluated on 50 trajectories.
• Figure 4: Optimal neighbourhood length exists and is dependent on the reservoir size. The mean valid time is shown for a given number of nodes $N$ and neighbourhood length ${l}$ for $M=32$ parallel reservoirs (see appendix Fig. \ref{['fig:ParRes_nei_all']} for qualitatively similar results of other numbers of parallel reservoirs). Best-performing neighbourhood lengths for $N>200$ nodes are in $[5\Delta x, 8\Delta x]$. Valid time declines sharply when the neighbourhood length is smaller than the optimal value.
• Figure 5: Autocorrelation function of the KSE has large magnitudes for short distances and decays towards zero for large distances. The wave-like spatial structure of the system (compare Fig. \ref{['fig:ks_prediction']}), induces alternations between high positive and negative values of spatial correlation. The first zero crossing is at a distance of $\approx 4.6\Delta x$ and the first minimum at $\approx 8.3\Delta x$.