Sequential Reservoir Computing for Efficient High-Dimensional Spatiotemporal Forecasting

TL;DR

This work addresses the computational bottlenecks of recurrent models for high-dimensional spatiotemporal forecasting by introducing Sequential Reservoir Computing (Sequential RC), a hierarchical RC architecture that stacks small reservoirs in sequence. By fixing reservoir weights and training only a ridge-readout, Sequential RC preserves long-term temporal dependencies while dramatically reducing memory and computational demands compared with LSTMs and standard RC. Across Lorenz63, 2D vorticity, and shallow-water datasets, Sequential RC achieves longer valid forecast horizons, improved structural similarity and PSNR, and up to three orders of magnitude lower training cost, demonstrating strong scalability for real-time scientific forecasting. The approach offers a practical path toward energy-efficient, high-fidelity forecasts in complex dynamical systems with high dimensionality.

Abstract

Forecasting high-dimensional spatiotemporal systems remains computationally challenging for recurrent neural networks (RNNs) and long short-term memory (LSTM) models due to gradient-based training and memory bottlenecks. Reservoir Computing (RC) mitigates these challenges by replacing backpropagation with fixed recurrent layers and a convex readout optimization, yet conventional RC architectures still scale poorly with input dimensionality. We introduce a Sequential Reservoir Computing (Sequential RC) architecture that decomposes a large reservoir into a series of smaller, interconnected reservoirs. This design reduces memory and computational costs while preserving long-term temporal dependencies. Using both low-dimensional chaotic systems (Lorenz63) and high-dimensional physical simulations (2D vorticity and shallow-water equations), Sequential RC achieves 15-25% longer valid forecast horizons, 20-30% lower error metrics (SSIM, RMSE), and up to three orders of magnitude lower training cost compared to LSTM and standard RNN baselines. The results demonstrate that Sequential RC maintains the simplicity and efficiency of conventional RC while achieving superior scalability for high-dimensional dynamical systems. This approach provides a practical path toward real-time, energy-efficient forecasting in scientific and engineering applications.

Figures (10)

• Figure 1: The schematic configuration of RC model architecture. The left-side blue nodes and right-side red nodes represent input and target variables, respectively.
• Figure 2: The schematic configuration of RC model architecture. The left-side blue nodes and right-side red nodes represent input and target variables, respectively. The outputs of each reservoir layer along with input data to the readout layer.
• Figure 3: Forecasts based on 2,000 training samples. The y-axis in each subplot represents the respective variable's value and the x-axis represents the lead-time of forecast in arbitrary time units.
• Figure 4: Comparison of Lorenz attractors for RNN (left column in blue), LSTM (second column from left in cyan), RC (third column from left in red), and Sequential RC (fourth column from left in magenta) and groundtruth (far right column in green). The Lorenz63 attractors illustrate the dynamic behavior of the systems, with differences in trajectory patterns and stability. The RNN is not able to follow the trajectory with any number of training samples. LSTM can't learn the trajectory but is able to pick the trajectory with 5,000 and 10,000 samples. RC and Sequential RC both are capable of forecasting trajectory in all training sample sizes.
• Figure 5: Return map for the $z$-dimension of the Lorenz63 system. The plot illustrates that RC and sequential RC models closely match the ground truth, showcasing their robust performance in all training sample sizes. In contrast, the RNN exhibits divergence, while the LSTM aligns closely but doesn't fully capture the dynamics. The comparison underscores the effectiveness of RC and sequential RC in maintaining stability while capturing the Lorenz63 dynamics.