Attractor learning for spatiotemporally chaotic dynamical systems using echo state networks with transfer learning

TL;DR

This work demonstrates that echo state networks, when equipped with transfer learning, can reliably predict the long-term statistical properties of spatiotemporally chaotic PDEs like the generalized Kuramoto–Sivashinsky equation and adapt efficiently to abrupt parameter changes in domain length and dispersion. By focusing on statistics such as the time-averaged power spectrum and attractor dimension, the authors quantify how TL updates to the ESN output layer enable accurate cross-regime predictions with relatively small additional data. They show both synthesis of KS/gKS statistics in fixed-parameter regimes and robust transfer across parameter jumps, including substantial improvements in single-trajectory prediction horizons. The results suggest practical potential for TL-augmented ESNs in tracking parametric chaos in real-world dynamical systems and motivate further exploration of data-efficient ML methods for chaotic PDEs.

Abstract

In this paper, we explore the predictive capabilities of echo state networks (ESNs) for the generalized Kuramoto-Sivashinsky (gKS) equation, an archetypal nonlinear PDE that exhibits spatiotemporal chaos. Our research focuses on predicting changes in long-term statistical patterns of the gKS model that result from varying the dispersion relation or the length of the spatial domain. We use transfer learning to adapt ESNs to different parameter settings and successfully capture changes in the underlying chaotic attractor. Previous work has shown that transfer learning can be used effectively with ESNs for single-orbit prediction. The novelty of our paper lies in our use of this pairing to predict the long-term statistical properties of spatiotemporally chaotic PDEs. We also show that transfer learning nontrivially improves the length of time that predictions of individual gKS trajectories remain accurate.

Figures (7)

• Figure 1: Once trained on KS trajectories from the spatial domain $[0,L]$, the ESN then accurately predicts the time-averaged power spectrum of the KS equation with the same spatial domain. Each blue curve shows the power spectrum obtained from direct numerical simulation, averaged over $10$ DNS trajectories used for prediction. Red curves show the ESN predictions of the power spectrum. Importantly, the ESN predictions of the power spectrum remain accurate long after ESN predictions of individual trajectories become inaccurate. Spatial domain sizes $L=22$ (top left), $L=29$ (top right), $L=35$ (bottom left), and $L=43$ (bottom right) correspond to four different numbers of linearly unstable Fourier modes (see Table \ref{['tab:L_data']}). Plots show $\log (e_{k})$ versus $\log (k)$, where $k$ denotes wavenumber.
• Figure 2: Transfer learning allows the ESN to accurately predict the time-averaged power spectrum of the KS equation as the size of the spatial domain varies. Curves illustrate power spectra. DNS-L43: Power spectrum obtained by averaging over the $10$ DNS trajectories in the $L=43$ regime used for prediction. ESN-L22: Power spectrum for the $L=22$ regime predicted by the ESN, once it is trained on $20$ trajectories from the $L=22$ regime. ESN-TL: Power spectrum for the $L=43$ regime predicted by the ESN after transfer learning of level $10\%$ (left), $25\%$ (middle), and $50\%$ (right). Plots show $\log (e_{k})$ versus $\log (k)$, where $k$ denotes wavenumber.
• Figure 3: Difference between the ESN-predicted KS trajectory and the KS trajectory computed by direct numerical simulation without transfer learning (top row) and with transfer learning (bottom row). Left and right columns correspond to Trajectory 2 for $L=25$ and $L=28$, respectively, in Table \ref{['tab:single_orbit']}.
• Figure 4: Once trained, the ESN accurately predicts the time-averaged power spectrum of the gKS equation. Left: $\gamma = 0$. Right: $\gamma = 0.1$. Each blue curve shows the power spectrum obtained by averaging over the $10$ DNS trajectories (each of length $T=10000$) used for prediction. Each red curve shows the power spectrum predicted by the ESN, after the untrained ESN has been trained using $20$ trajectories (each of length $T=10000$). Plots show $\log (e_{k})$ versus $\log (k)$, where $k$ denotes wavenumber. Spatial domain: $L=43$.
• Figure 5: Transfer learning allows the ESN to accurately predict the time-averaged power spectrum of the gKS equation as the dispersion parameter $\gamma$ instantaneously jumps. Blue curve: Power spectrum of the gKS equation in the $\gamma = 0.1$ regime obtained by averaging over the $10$ DNS trajectories (each of length $T=10000$) used for prediction. Green curve: Power spectrum predicted by the ESN after a first round of training using $20$ trajectories from the $\gamma = 0$ regime (each of length $T=10000$) and then a training update using TL data from the $\gamma = 0.1$ regime at level $10\%$. Yellow curve: Power spectrum predicted by the ESN after training only on $20$ trajectories from the $\gamma = 0$ regime (no transfer learning). Red curve: Power spectrum predicted by an ESN that has been trained only on the TL data, namely $2$ trajectories from the $\gamma = 0.1$ regime. Plots show $\log (e_{k})$ versus $\log (k)$, where $k$ denotes wavenumber. Spatial domain: $L=43$.