mLaSDI: Multi-stage latent space dynamics identification

TL;DR

mLaSDI introduces a multi-stage residual-learning extension to Latent Space Dynamics Identification, addressing the trade-off between accurate data reconstruction and interpretable latent dynamics. By training successive decoders to model residuals while sharing the same latent space, it preserves interpretability and dramatically improves reconstruction and prediction accuracy on challenging, high-frequency problems. Across multiscale oscillations, unsteady wake flow, and 1D-1V Vlasov simulations, mLaSDI achieves order-of-magnitude gains in error reduction and often requires less training time than standard LaSDI variants. The approach offers a practical pathway to more accurate, non-intrusive ROMs with interpretable latent dynamics for complex parametric PDEs.

Abstract

Accurately solving partial differential equations (PDEs) is essential across many scientific disciplines. However, high-fidelity solvers can be computationally prohibitive, motivating the development of reduced-order models (ROMs). Recently, Latent Space Dynamics Identification (LaSDI) was proposed as a data-driven, non-intrusive ROM framework. LaSDI compresses the training data via an autoencoder and learns user-specified ordinary differential equations (ODEs), governing the latent dynamics, enabling rapid predictions for unseen parameters. While LaSDI has produced effective ROMs for numerous problems, the autoencoder must simultaneously reconstruct the training data and satisfy the imposed latent dynamics, which are often competing objectives that limit accuracy, particularly for complex or high-frequency phenomena. To address this limitation, we propose multi-stage Latent Space Dynamics Identification (mLaSDI). With mLaSDI, we train LaSDI sequentially in stages. After training the initial autoencoder, we train additional decoders which map the latent trajectories to residuals from previous stages. This staged residual learning, combined with periodic activation functions, enables recovery of high-frequency content without sacrificing interpretability of the latent dynamics. Numerical experiments on a multiscale oscillating system, unsteady wake flow, and the 1D-1V Vlasov equation demonstrate that mLaSDI achieves significantly lower reconstruction and prediction errors, often by an order of magnitude, while requiring less training time and reduced hyperparameter tuning compared to standard LaSDI.

Figures (11)

• Figure 1: Schematic of mLaSDI. The first stage learns an autoencoder which is trained to reconstruct the data while learning SINDy dynamics in the latent space. The latent trajectories (solid black lines) are approximated using SINDy (dashed red lines). In the second stage, a new decoder maps the SINDy-approximated latent trajectories to the normalized residual from the first stage. The final reconstruction combines outputs from both decoders.
• Figure 2: Applying GPLaSDI to toy problem \ref{['eq:multistage_func']}. (Left) Loss for GPLaSDI and (Right) Latent space trajectories of autoencoder (solid lines) and SINDy approximation (dashed lines) for training case $A = 1.4$.
• Figure 3: Applying GPLaSDI and mLaSDI with 2 stages to \ref{['eq:multistage_func']} with $A = 1.4$. (Left) Reconstruction of the training data for both methods. (Right) Results after subtracting the sine wave from the approximations to visualize how well each method approximates the small, high-frequency cosine wave. Note that the data and mLaSDI are nearly overlapping.
• Figure 4: Relative error for GPLaSDI and mLaSDI applied to toy problem \ref{['eq:multistage_func']}. GPLaSDI is trained for 50,000 iterations while mLaSDI has 2 stages each trained for 10,000 iterations. Both methods are trained on the data from cases $A = (1.0, 1.4)$, and we predict for $A = 1.2$.
• Figure 5: Mesh used to solve unsteady wake flow, rotated 90 degrees clockwise.