Generative Learning for Slow Manifolds and Bifurcation Diagrams

TL;DR

The paper tackles efficient initialization on slow manifolds and across bifurcation surfaces in multiscale dynamical systems by leveraging conditional score-based diffusion models (cSGMs). It integrates diffusion-map–based manifold learning and Geometric Harmonics to identify low-dimensional coordinates and lift samples back to the ambient space, enabling fast, QoI-conditioned sampling and reconstruction of steady states on bifurcation diagrams. Key contributions include a data-driven framework for training cSGMs on manifolds, a lifting strategy to reduce noise, and demonstrations on cusp bifurcations, the Chafee–Infante inertial manifold, and plug-flow reactor PDEs with non-uniform sampling considerations. The approach offers a flexible, scalable pathway to accelerate multiscale simulations, interpolate missing segments of bifurcation diagrams, and recover full-field PDE solutions from reduced-order representations.

Abstract

In dynamical systems characterized by separation of time scales, the approximation of so called ``slow manifolds'', on which the long term dynamics lie, is a useful step for model reduction. Initializing on such slow manifolds is a useful step in modeling, since it circumvents fast transients, and is crucial in multiscale algorithms alternating between fine scale (fast) and coarser scale (slow) simulations. In a similar spirit, when one studies the infinite time dynamics of systems depending on parameters, the system attractors (e.g., its steady states) lie on bifurcation diagrams. Sampling these manifolds gives us representative attractors (here, steady states of ODEs or PDEs) at different parameter values. Algorithms for the systematic construction of these manifolds are required parts of the ``traditional'' numerical nonlinear dynamics toolkit. In more recent years, as the field of Machine Learning develops, conditional score-based generative models (cSGMs) have demonstrated capabilities in generating plausible data from target distributions that are conditioned on some given label. It is tempting to exploit such generative models to produce samples of data distributions conditioned on some quantity of interest (QoI). In this work, we present a framework for using cSGMs to quickly (a) initialize on a low-dimensional (reduced-order) slow manifold of a multi-time-scale system consistent with desired value(s) of a QoI (a ``label'') on the manifold, and (b) approximate steady states in a bifurcation diagram consistent with a (new, out-of-sample) parameter value. This conditional sampling can help uncover the geometry of the reduced slow-manifold and/or approximately ``fill in'' missing segments of steady states in a bifurcation diagram.

Figures (13)

• Figure 1: Data produced by the cusp bifurcation equations shown as points with its 2D projection to the control parameter space below the samples. The black points (and shaded region of the projection) correspond to the values of $\mu$ and $\lambda$ for which there exist three steady states, and the blue points (and unshaded region of the projection) correspond to the parameter values at which there is a single steady state.
• Figure 2: cSGM: A slice of training data where $\mu \approx 0$ (black points) plotted with three-dimensional outputs from a cSGM (red points) where the conditioned values are $\mu = 0$ (left) and $\mu = 0, \lambda = 2$ (right).
• Figure 3: MCS-based cSGM: A slice of training data where $\mu \approx 0$ (black points) plotted with three-dimensional outputs from the MCS-based cSGM (red points) where the conditioned values are $\mu = 0$ (left) and $\mu = 0, \lambda = 2$ (right).
• Figure 4: In both plots, the first three Fourier modes, $\alpha_1$, $\alpha_2$, and $\alpha_3$ are plotted on the x, y, and z axes respectively and are colored by $\alpha_1$. On the left, a 10-D (all ten Fourier modes) cSGM output of 1000 samples conditioned at $\alpha_1 = 0$ is plotted over the manifold as black points. On the right, a 2-D (only the first two Fourier modes) cSGM output of 1000 samples conditioned at $\alpha_1 = 0$ is plotted over the manifold as black points.
• Figure 5: On the left, the first two Fourier modes $\alpha_1$ and $\alpha_2$ are plotted on the x and y axes respectively and are colored by $\alpha_1$. A cSGM output of 1000 samples of all ten Fourier modes conditioned at $\alpha_1 = 0$ is plotted over the manifold as black points. Three points from this generated set are colored red, blue, and green. On the right, three reconstructed $u(x,t)$ profiles corresponding to the three colored dots in the left plot are shown.