Next-Generation Reservoir Computing for Dynamical Inference

TL;DR

The paper introduces a scalable NGRC framework that uses a pseudorandom nonlinear projection of time-delay embedded inputs to create a high-dimensional feature space, enabling a simple linear readout to predict dynamical systems. It demonstrates robust short-term forecasting, attractor reconstruction, bifurcation mapping, and asymptotic phase recovery from partial and noisy data, with training-time measurement noise acting as an implicit regularizer that enhances long-horizon stability. The approach emphasizes interpretability and controllability, offering a transparent alternative to traditional RC with potential for digital twins and surrogate modeling across physical, biological, and financial domains. Limitations include extrapolation beyond training regimes, but the method shows strong generalization within learned regions and avenues for future network-inference extensions.

Abstract

We present a simple and scalable implementation of next-generation reservoir computing (NGRC) for modeling dynamical systems from time-series data. The method uses a pseudorandom nonlinear projection of time-delay embedded inputs, allowing the feature-space dimension to be chosen independently of the observation size and offering a flexible alternative to polynomial-based NGRC projections. We demonstrate the approach on benchmark tasks, including attractor reconstruction and bifurcation diagram estimation, using partial and noisy measurements. We further show that small amounts of measurement noise during training act as an effective regularizer, improving long-term autonomous stability compared to standard regression alone. Across all tests, the models remain stable over long rollouts and generalize beyond the training data. The framework offers explicit control of system state during prediction, and these properties make NGRC a natural candidate for applications such as surrogate modeling and digital-twin applications.

Figures (12)

• Figure 1: Comparison of traditional Reservoir Computing (RC) Ott2018 and Next Generation Reservoir Computing (NGRC) NGRC2021 schemes.
• Figure 2: Schematic of the NGRC one-step predictor. A normalized, time-delay embedded state $\mathbf{u}_H(t)$ is lifted by the nonlinear projection $P$ into a high-dimensional feature vector, which is mapped linearly by $\mathbf{W}_\text{out}$ to predict $\mathbf{u}(t+\tau)$. The diagram shows the single scalar signal $\mathbf{u}(t)$ case; for $N$ observables, $\mathbf{u}_H$ has dimension $NH$ and $\mathbf{W}_\text{out}$ is an $M \times N$ matrix.
• Figure 3: Distribution of $p_m$ values generated by the projection $P$ for a long time series of the Lorenz system \ref{['lorenz']}. The values remain bounded within the unit interval and are smoothly distributed, reflecting the stable and well-scaled nature of the projection scheme.
• Figure 4: Mean-squared one-step prediction error as a function of the projection dimension $M$ under three regularization scenarios: $(a)$ no regularization, $(b)$ Tikhonov regularization with $\lambda = 0.01$, and $(c)$ addition of 1% Gaussian measurement noise to the training inputs. $(a)$ Without regularization, errors initially decrease but instabilities appear for larger $M$. $(b)$ Tikhonov regularization suppresses high-$M$ instabilities, but mild overfitting persists, with training errors consistently below validation errors. $(c)$ Adding measurement noise acts as an implicit regularizer by broadening the sampled state space, yielding training and validation errors that remain comparable across the full range of $M$. This motivates the use of 1% measurement noise in subsequent experiments as a practical balance between stability and predictive accuracy. Excessive noise, however, degrades performance, so a trade-off applies. All results use long trajectories ($T = 10^5$) for both training and validation.
• Figure 5: Long-term autonomous prediction test for NGRC under three regularization settings: $(a)$ Tikhonov regularization with $\lambda=0.01$ and no measurement noise, $(b)$ the same Tikhonov regularization with $0.1\%$ measurement noise added to the training inputs, and $(c)$$1\%$ measurement noise without Tikhonov regularization. For each setting, 100 independently trained models are rolled out from 10 slightly perturbed initial conditions, and trajectories are compared with a reference solution of the true ODE system \ref{['lorenz']}. Because all signals are normalized to $[0,1]$, a trajectory is considered divergent once it leaves this interval, at which point the rollout is terminated. The fraction of runs that diverged is shown alongside the trajectories. $(a)$ sees rapid transverse instability and early divergence in most runs; $(b)$ shows substantially reduced escape rates; $(c)$ demonstrates the strongest stability, with no trajectories leaving escaping over the displayed time window.