Prediction of Unobserved Bifurcation by Unsupervised Extraction of Slowly Time-Varying System Parameter Dynamics from Time Series Using Reservoir Computing

TL;DR

The model demonstrated robust predictive performance, showing that the reservoir computing framework can handle nonlinear, non-stationary systems without prior knowledge of the system's true parameters.

Abstract

Nonlinear and non-stationary processes are prevalent in various natural and physical phenomena, where system dynamics can change qualitatively due to bifurcation phenomena. Traditional machine learning methods have advanced our ability to learn and predict such systems from observed time series data. However, predicting the behavior of systems with temporal parameter variations without knowledge of true parameter values remains a significant challenge. This study leverages the reservoir computing framework to address this problem by unsupervised extraction of slowly varying system parameters from time series data. We propose a model architecture consisting of a slow reservoir with long timescale internal dynamics and a fast reservoir with short timescale dynamics. The slow reservoir extracts the temporal variation of system parameters, which are then used to predict unknown bifurcations in the fast dynamics. Through experiments using data generated from chaotic dynamical systems, we demonstrate the ability to predict bifurcations not present in the training data. Our approach shows potential for applications in fields such as neuroscience, material science, and weather prediction, where slow dynamics influencing qualitative changes are often unobservable.

Figures (9)

• Figure 1: Schematic diagram of the numerical examination discussed in Section \ref{['esf']}. The observation signal $y(n)$ is generated by a nonlinear system $\mathrm{d}x/\mathrm{d}t = f_x (\bm{x}; \lambda)$. The slow reservoir consists of leaky neuron models with a very long leak rate, and the spectral radius of the recurrent connection is set to 1.
• Figure 2: Schematic diagram showing the open-loop training phase and the closed-loop prediction phase. (A) In the training phase, observation $y(n)$ is fed to both the slow and fast reservoirs. The output of the slow reservoir is also input to the fast reservoir. Additionally, the output of the slow reservoir is input to the "slow dynamics predictor" reservoir. After the training phase, the output weights of both the slow dynamics predictor and the fast reservoir are optimized to conduct one-step-ahead prediction of their own inputs. (B) In the prediction phase, feedback loops are added to the slow dynamics predictor and the fast reservoir to make the whole system a single autonomous dynamical system that can predict time series of $y(n)$.
• Figure 3: Response of the slow reservoir to time series generated by the Lorenz system in the Experiment 1. (A) True parameter value $\lambda$ of the Lorenz system slowly changing from $\lambda = 64$ to $\lambda=100$.(B) Variable $y(n) = x_1(\Delta t \cdot n)$, representing the first element of the state of the Lorenz system used as the input to the reservoir. (C) Local minima and maxima of the trace shown in (B). (D, E) Values of the internal states, $x_i$, of the slow reservoir characterized by rapid and slow temporal fluctuations, respectively. (F) Extracted slow dynamics calculated as the average of the absolute values of internal nodes exhibiting slow behavior. All panels are plotted against time in the horizontal axis.
• Figure 4: Response of the slow reservoir to time series generated by the Lorenz system in the Experiment 1 with an expanded time axis. This figure shows the same data as in fig. \ref{['Result_Lorenz']} but with an expanded time scale. (A) True parameter value $\lambda$ of the Lorenz system. (B) Variable $y(n) = x_1(\Delta t \cdot n)$, representing the first element of the state of the Lorenz system used as the input to the reservoir. (C) Local minima and maxima of the trace shown in (B). (D,E) Values of the internal states, $x_i$, of the slow reservoir characterized by rapid and slow temporal fluctuations, respectively. (F) Extracted slow dynamics calculated as the average of the absolute values of internal nodes exhibiting slow behavior. All panels are plotted against time in the horizontal axis.
• Figure 5: Response of the slow reservoir to time series generated by the Rössler equation. (A) True parameter value $\lambda$ of the Rössler equation. (B) Variable $y(n) = x_1(\Delta t \cdot n)$, representing the first element of the state of the Rössler equation used as the input to the reservoir. (C) Local minima and maxima of the trace shown in (B). (D, E) Values of the internal states, $x_i$, of the slow reservoir characterized by rapid and slow temporal fluctuations, respectively. (F) Extracted slow dynamics calculated as the average of the absolute values of internal nodes exhibiting slow behavior. All panels are plotted against time in the horizontal axis.