Learn one size to infer all: Exploiting translational symmetries in delay-dynamical and spatio-temporal systems using scalable neural networks

TL;DR

The paper tackles predicting high-dimensional dynamical systems that exhibit translational symmetry, proposing scalable reservoir-based networks that learn from a single system size. By encoding temporal and spatial translational symmetry, the authors show that the networks can be resized to infer untrained dynamics across different delays and spatial extensions, effectively reconstructing entire bifurcation diagrams. They demonstrate this through Mackey-Glass and Ikeda delay systems and a Kuramoto-Sivashinsky spatio-temporal model, including delay estimation, multistability inference, and weather-climate behavior distinctions. The approach offers data-efficient, symmetry-guided predictions with potential applications when parameter settings are inaccessible, contributing to sustainable and scalable machine learning for complex dynamical systems.

Abstract

We design scalable neural networks adapted to translational symmetries in dynamical systems, capable of inferring untrained high-dimensional dynamics for different system sizes. We train these networks to predict the dynamics of delay-dynamical and spatio-temporal systems for a single size. Then, we drive the networks by their own predictions. We demonstrate that by scaling the size of the trained network, we can predict the complex dynamics for larger or smaller system sizes. Thus, the network learns from a single example and, by exploiting symmetry properties, infers entire bifurcation diagrams.

Figures (9)

• Figure 1: Scheme of the delayed echo state network. The reservoir contains delayed internode weighted connections (green), where the delay $D$ of the nodes connections can be adjusted.
• Figure 2: Two-dimensional projection (x-axis $y(t)$, y-axis $y(t-\tau)$) of attractors of the chaotic Mackey-Glass system for different delay lengths in a), b), c), & d). e) Bifurcation diagram generated using the Mackey-Glass delay system. Inferred attractors by the dESN trained on a single example of the Mackey-Glass system with $\tau=100$ shown in f), g), h) & i). j) Bifurcation diagram inferred by the dESN trained on data of a Mackey-Glass system with $\tau=100$.
• Figure 3: a) Time series of the original Mackey Glass system (blue) and the autonomously continued (green) chaotic attractor using a dESN. b) Divergence rate of the chaotic system and the time series generated by the dESN (green) as shown in a), the orange line indicates the average divergence rate of 20 different dESNs initialized at 20 different trajectories of the chaotic Mackey-Glass system. The blue line indicates the divergence rate related to the largest Lyapunov exponent $\lambda=0.009$ of the Mackey-Glass system with $\tau=30$. In c) and d), two-dimensional projection of the chaotic attractor of the Mackey-Glass system with a delay of $\tau=30$ and the dESN prediction with $D=30$, respectively.
• Figure 4: The dynamics of the spatio-temporal Kuramoto-Sivashinsky system $y(x,n)$ (top row), the predicted dynamics from a parallel network architecture $\hat{y}(x,n)$ (mid row), and their difference $y(x,n)-\hat{y}(x,n)$(bottom row). The parallel networks are trained with data from the Kuramoto-Sivashinsky system of spatial extension $L=10\pi$ (red box), by adapting the network architecture it generates the dynamics also for smaller and larger spatial extensions (yellow box). For comparison both systems were initialized with the same initial conditions.
• Figure 5: Estimating the delay a from time series of the chaotic Mackey-Glass system for three different delay lengths $\tau$. The red line marks the delay $\tau$ that underlies the data set a) $\tau=30$, b) $\tau=60$ and c) $\tau=100$. The blue dots mark the one-step-ahead prediction accuracy of dESNs with different delays $D$ (x-axis) and different sets of hyperparameters. The accuracy is determined by the NRMSE (normalized root mean square error) of the one step ahead prediction.