Generative Learning for Forecasting the Dynamics of Complex Systems

TL;DR

G-LED addresses the challenge of fast, accurate forecasting for high dimensional, multiscale systems by learning a low dimensional latent macro space and using a diffusion based decoder to lift to full states. The method combines a non trainable encoder to obtain z_t from s_t, a multi head auto regressive attention model to evolve z_t over macro time steps, and a diffusion model that incorporates physics through virtual observations to reconstruct s_t. The framework is validated on KS, 2D flow over a backward facing step, and 3D turbulent channel flow, demonstrating faithful statistics such as energy spectra and Reynolds stresses while delivering substantial online speedups. This work provides a scalable, physics informed surrogate approach with potential for parametric extensions and improved latent dimension selection.

Abstract

We introduce generative models for accelerating simulations of complex systems through learning and evolving their effective dynamics. In the proposed Generative Learning of Effective Dynamics (G-LED), instances of high dimensional data are down sampled to a lower dimensional manifold that is evolved through an auto-regressive attention mechanism. In turn, Bayesian diffusion models, that map this low-dimensional manifold onto its corresponding high-dimensional space, capture the statistics of the system dynamics. We demonstrate the capabilities and drawbacks of G-LED in simulations of several benchmark systems, including the Kuramoto-Sivashinsky (KS) equation, two-dimensional high Reynolds number flow over a backward-facing step, and simulations of three-dimensional turbulent channel flow. The results demonstrate that generative learning offers new frontiers for the accurate forecasting of the statistical properties of complex systems at a reduced computational cost.

Figures (25)

• Figure 1: Summary of G-LED (Generative Learning for the effective dynamics): The initial condition, a high-dimensional micro state, is encoded with a non-trainable downsampler to its corresponding macro state. Then, the dynamics are evolved in the low-dimensional space by a multi-head, auto-regressive attention model. All obtained macro states are decoded to their micro states by the spatiotemporal Bayesian diffusion model using the reverse process of the diffusion model and potentially additional physical information.
• Figure 2: Results obtained with G-LED (left) and numerical simulation (right) on different test trajectories (from top to bottom). The horizontal direction depicts the time $t \in[0,100]$ and the vertical direction corresponds to the spatial coordinate $x \in[0,22]$.
• Figure 3: The error ($e(t)=\frac{||{\bm{s}}_t^{\mathrm{numerical}} - {\bm{s}}_t^{\mathrm{LED}}||^2_2}{||{\bm{s}}_t^{\mathrm{numerical}}||^2_2}$) over all testing sequences (left); The density of values in the $u_x - u_{xx}$ space computed from all testing sequences from G-LED (middle) and numerical simulations (right).
• Figure 4: Geometry of flow domain (solid lines), area of interest (shadow zone) for prediction, and boundary conditions of inlet (\ref{['line:cyl0:inlet']}), outlet (\ref{['line:cyl0:out']}), and no-slip wall (\ref{['line:cyl0:wall']}) for the case of flow over backward-facing step.
• Figure 5: The streamwise velocity rollout of G-LED (left) and LES (right) given the same initial state at $t=0.05, 0.50, 0.95, 1.25$ (from top to bottom)