DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems

TL;DR

DySLIM addresses the difficulty of learning chaotic, dissipative dynamics by targeting the attractor's invariant measure $\mu^*$ in addition to the local dynamics. It regularizes standard trajectory-misfit objectives with a measure-distance term, implemented via Maximum Mean Discrepancy (MMD) between $\mu^*$ and the learned invariant measure $({\mathcal{S}}_\theta^\ell)_{\#}\mu^*$, and estimates the regularizer through time-stepping. The method supports unconditional and conditional regularization and combines with existing objectives to yield stable long-horizon behavior while preserving short-term accuracy. Across Lorenz 63, Kuramoto–Sivashinsky, and Kolmogorov flow, DySLIM improves long-term statistical fidelity (lower SD, MELR, covRMSE) and often enhances short-term trajectory predictions, suggesting practical utility for complex systems with slowly varying distributions such as weather and climate models.

Abstract

Learning dynamics from dissipative chaotic systems is notoriously difficult due to their inherent instability, as formalized by their positive Lyapunov exponents, which exponentially amplify errors in the learned dynamics. However, many of these systems exhibit ergodicity and an attractor: a compact and highly complex manifold, to which trajectories converge in finite-time, that supports an invariant measure, i.e., a probability distribution that is invariant under the action of the dynamics, which dictates the long-term statistical behavior of the system. In this work, we leverage this structure to propose a new framework that targets learning the invariant measure as well as the dynamics, in contrast with typical methods that only target the misfit between trajectories, which often leads to divergence as the trajectories' length increases. We use our framework to propose a tractable and sample efficient objective that can be used with any existing learning objectives. Our Dynamics Stable Learning by Invariant Measure (DySLIM) objective enables model training that achieves better point-wise tracking and long-term statistical accuracy relative to other learning objectives. By targeting the distribution with a scalable regularization term, we hope that this approach can be extended to more complex systems exhibiting slowly-variant distributions, such as weather and climate models.

Figures (18)

• Figure 1: Improved stability with regularized DySLIM objectives in the Kuramoto–Sivashinsky (KS) and Lorenz 63 systems. (a) Sample ground truth and predicted trajectory across models trained on the KS system using Curriculum training (Curr) and the Pushforward trick (Pfwd) with/without regularization. The base versions showcase the blow-up issue (Curr) and wrong long-time dynamics (Pfwd). (b) Sinkhorn Divergence (SD; $\downarrow$) between trajectories at various rollout times of the Lorenz 63 system. Each point represents a random training seed, with the solid line indicating median values. (c) Cosine similarity ($\uparrow$) over time for the Lorenz 63 system. Each line corresponds to one of five random training seeds with bolded lines indicating median values.
• Figure 2: Regularized DySLIM objectives outperform baselines for the KS system. (a) Cosine similarity ($\uparrow$) over time. Each line corresponds to the mean over trajectories of each of five random training seeds, with bold lines indicating median values. (b) Sinkhorn Divergence (SD; $\downarrow$) between trajectories at various rollout times. Each point represents a random training seed that remains stable, with the solid line indicating median values.
• Figure 3: (Left) Sample reference and predicted trajectory across models trained on the Kolmogorov Flow data using the Curr and Pfwd objectives, together with the regularized versions. (Right) Evolution of the cosine similarity over time for Curr and Pfwd objectives with and without regularization. The solid line is the median among $160$ runs ($32$ trajectories for each of the $5$ random seeds), and the shaded regions correspond to the second and third quartile. ($\lambda_1 = 0$, $\lambda_2 = 100$, batch size = 128 and learning rate = $5\mathrm{e}^{-4})$.
• Figure 4: Histograms of trajectories at rollout time $t=400$ for one of the random training seeds. We showcase the well known "butterfly" attractor.
• Figure 5: Values of the MMD metric for the baselines and the DySLIM regularization. Each point represents a random training seed that remains stable, with the solid line indicating median values.