Unsupervised learning for anticipating critical transitions

TL;DR

The power of the unsupervised learning scheme is demonstrated using prototypical dynamical systems including the spatiotemporal Kuramoto-Sivashinsky system and the scheme can be extended to scenarios where the target system is driven by several independent parameters or with partial state observations.

Abstract

For anticipating critical transitions in complex dynamical systems, the recent approach of parameter-driven reservoir computing requires explicit knowledge of the bifurcation parameter. We articulate a framework combining a variational autoencoder (VAE) and reservoir computing to address this challenge. In particular, the driving factor is detected from time series using the VAE in an unsupervised-learning fashion and the extracted information is then used as the parameter input to the reservoir computer for anticipating the critical transition. We demonstrate the power of the unsupervised learning scheme using prototypical dynamical systems including the spatiotemporal Kuramoto-Sivashinsky system. The scheme can also be extended to scenarios where the target system is driven by several independent parameters or with partial state observations.

Figures (8)

• Figure 1: Integrated architecture of VAE and parameter-driven reservoir computing. The VAE consists of the encoder and the decoder, with the latent distribution parameter extracted from the input data in the bottleneck, and it identifies the bifurcation parameter and extracts its variations. The input to the reservoir computer consist of the time-series data from the target system and the VAE-extracted parameter variations, and the output is the prediction of long-term dynamics.
• Figure 2: Principle of VAE identification of the bifurcation parameters. The VAE designates latent-parameter channels whose number is larger than the actual number of the bifurcation parameters. With the time series data from distinct bifurcation-parameter values as the input to the VAE, each latent parameter channel produces values that follow a statistical distribution with mean $\mu_z$ and variance $\sigma^2_z$. Shown are schematic probability distributions of (a) $\mu_z$ and (b) $\sigma^2_z$, which correspond, respectively, to the true bifurcation parameter (light blue) and a "false" latent parameter that does not correspond to any actual parameter (dark blue).
• Figure 3: Unsupervised-learning based anticipation of a critical transition in the chaotic Lorenz system with $\rho$ as the single bifurcation parameter for $\beta = 8/3$ and $\sigma = 10$. (a) Bifurcation diagram. The VAE is trained with time series from the interval specified by the two vertical blue dashed lines. (b) VAE identification of the bifurcation parameter according to the behaviors of the variance of the mean $\mu_z$ of the latent parameter (blue) and the mean of its variance $\sigma^2_z$ (red) for the five parameter channels in the VAE. (c) VAE-detected parameter versus the ground truth physical parameter (green dots) and the linear fitting (solid yellow line). (d) Histogram of the predicted critical point $\rho^*$ obtained from $1000$ random realizations of the reservoir computer. The resulting distribution centers about the ground truth $\rho_c \approx 24.06$.
• Figure 4: Predicting critical transition in the nonlinear wave system described by the Kuramoto-Sivashinsky equation. Two examples of sustained and transient spatiotemporal chaos are shown for (a) $\phi = 199.94 \leq \phi_c$ and (b) $\phi = 200.14 \geq \phi_c$. (c) Variance of $\mu_z$ (blue) and mean of $\sigma^2_z$ (red) for the five parameter channels in the trained VAE. (d) VAE's detected parameters versus the ground truth parameter (green dots) and linear fitting (yellow solid line). (e) Histogram of the predicted critical point $\phi^*$ and the relative errors with respect to the ground truth $\phi_c \approx 200.04$. The distribution is collected from 210 random reservoir realizations.
• Figure 5: VAEs and parameter-driven reservoir computing. (a) Structure of a VAE. (b) Structure of parameter-driven reservoir computing, where the input consists of two components: time-series data and the values of the VAE-extracted bifurcation parameter. (c) Scheme of training for predicting critical transitions. The light blue region represents the regime of normal system operation while the light orange region denotes the parameter regime of collapse. A critical transition from normal operation to collapse occurs at the parameter value $p_c$, and $p_0 < p_c$ is the current operation point. Historical time-series data from a small number of parameter values in the normal regime are used for training, as indicated by the four vertical dashed green lines. Prediction is done for $p = p_0 + \Delta p$, where $\Delta p > 0$ is a parameter drift.