On latent dynamics learning in nonlinear reduced order modeling

TL;DR

This work develops latent dynamics models ($LDM$) for parameterized nonlinear time-dependent PDEs, formulating a time-continuous reduced-order framework that couples nonlinear dimensionality reduction with latent dynamics. It introduces three levels of approximation—time-continuous $LDM$, discrete $Δ ext{LDM}$, and learnable $Δ ext{LDM}_ heta$—and proves error, stability, and convergence properties, including a perfect embedding assumption to guide latent dimension. The DL realization uses a fully-convolutional autoencoder and an affinely-parameterized PNODE with sinusoidal time/parameter embeddings to inject $(t,oldsymbol{ u})$ into the latent dynamics, enabling time-continuous multi-query ROMs that can be queried at arbitrary times with bounded error. Numerical experiments on Burgers' and advection-diffusion-reaction problems demonstrate accurate time evolution, time-continuity under refined temporal grids, zero-stability under perturbations, and favorable parameter generalization, with substantial online speedups. This framework offers a mathematically rigorous, scalable path toward accurate, real-time surrogate models for complex, time-dependent PDEs in multi-query settings.

Abstract

In this work, we present the novel mathematical framework of latent dynamics models (LDMs) for reduced order modeling of parameterized nonlinear time-dependent PDEs. Our framework casts this latter task as a nonlinear dimensionality reduction problem, while constraining the latent state to evolve accordingly to an (unknown) dynamical system. A time-continuous setting is employed to derive error and stability estimates for the LDM approximation of the full order model (FOM) solution. We analyze the impact of using an explicit Runge-Kutta scheme in the time-discrete setting, resulting in the $Δ\text{LDM}$ formulation, and further explore the learnable setting, $Δ\text{LDM}_θ$, where deep neural networks approximate the discrete LDM components, while providing a bounded approximation error with respect to the FOM. Moreover, we extend the concept of parameterized Neural ODE - recently proposed as a possible way to build data-driven dynamical systems with varying input parameters - to be a convolutional architecture, where the input parameters information is injected by means of an affine modulation mechanism, while designing a convolutional autoencoder neural network able to retain spatial-coherence, thus enhancing interpretability at the latent level. Numerical experiments, including the Burgers' and the advection-reaction-diffusion equations, demonstrate the framework's ability to obtain, in a multi-query context, a time-continuous approximation of the FOM solution, thus being able to query the LDM approximation at any given time instance while retaining a prescribed level of accuracy. Our findings highlight the remarkable potential of the proposed LDMs, representing a mathematically rigorous framework to enhance the accuracy and approximation capabilities of reduced order modeling for time-dependent parameterized PDEs.

Figures (11)

• Figure 1: Different levels of approximation in the proposed LDM framework, illustrating: (i) the time-continuous full-order model ${\mathbf{u}}_h(t;{\boldsymbol{\mu}})$, the time-continuous LDM approximation $\tilde{{\mathbf{u}}}_h(t;{\boldsymbol{\mu}})$ with the associated latent dynamics ${\mathbf{u}}_n(t;{\boldsymbol{\mu}})$; (ii) the time-discrete approximation $\tilde{{\mathbf{u}}}_h^k({\boldsymbol{\mu}})$ arising from the numerical solution of the latent dynamics ${\mathbf{u}}_n^k({\boldsymbol{\mu}})$, leading to the notion of ${\Delta\text{LDM}}$; (iii) the learned approximation $\hat{{\mathbf{u}}}_h^k({\boldsymbol{\mu}})$ and the corresponding learned latent dynamics $\hat{{\mathbf{u}}}_n^k({\boldsymbol{\mu}})$, associated to the learnable ${\Delta\text{LDM}}_\theta$. Here, $\Psi$ and $\Psi'$ denote the nonlinear projection and reconstruction maps, respectively.
• Figure 2: Commutative diagram explaining the LDM scheme, approximating the mapping ${\mathbf{u}}_{0,h} \mapsto {\mathbf{u}}_h(t)$ defined by the full-order dynamics ${\mathbf{f}}_h$, by means of a latent dynamics ${\mathbf{f}}_n$ and the nonlinear projection and reconstruction maps $\Psi, \Psi'$.
• Figure 3: Error decomposition. Illustration of the upper bound referring to the error decomposition formula \ref{['eq:error_decomposition']} in the discrete case, for the full-order state approximation ${\tilde{\mathbf{u}}}_h^k$ provided by the ${\Delta\text{LDM}}$ scheme.
• Figure 4: Framework overview. The diagram summarizes the proposed framework, considering the learnable setting (${\Delta\text{LDM}}_\theta$), the time-continuous (LDM) and time-discrete (${\Delta\text{LDM}}$) approximations, together with the respective main properties.
• Figure 5: Affinely-parameterized latent dynamics architecture. Detailed structure of the proposed convolutional parameterized latent dynamics. The two main components -- the embedding module, which processes the time-parameters inputs, and the stack of affinely-modulated convolutional layers -- are highlighted.

Theorems & Definitions (12)

• Definition 2.1: Time-continuous approximation
• Definition 3.1: Latent dynamics problem
• Definition 3.2: Latent dynamics model
• Proposition 3.3
• Definition 3.4
• Theorem 3.5
• proof
• Remark 3.6
• Remark 4.1
• Remark 4.2: Temporal regularization