Sparse identification of nonlinear dynamics and Koopman operators with Shallow Recurrent Decoder Networks

TL;DR

The paper tackles the challenge of learning governing physics from high-dimensional spatio-temporal data by uniting sparse sensing with interpretable dynamics. It introduces SINDy-SHRED, which regularizes a SHRED-based latent space to follow a SINDy-class ODE (with Koopman-SHRED as a linear variant), enabling parsimonious, interpretable models that reconstruct full fields from sparse sensors. The approach yields provable convergence properties and a globally convex loss landscape, demonstrating superior long-term accuracy, data efficiency, and speed on diverse datasets including SST, turbulence, and GoPro-style video. This framework enables GoPro Physics-style discovery of new dynamics directly from real-time data streams while maintaining practical compute requirements. It offers a scalable path to physics-informed, data-efficient modeling across scientific and engineering domains.

Abstract

Modeling real-world spatio-temporal data is exceptionally difficult due to inherent high dimensionality, measurement noise, partial observations, and often expensive data collection procedures. In this paper, we present Sparse Identification of Nonlinear Dynamics with SHallow REcurrent Decoder networks (SINDy-SHRED), a method to jointly solve the sensing and model identification problems with simple implementation, efficient computation, and robust performance. SINDy-SHRED uses Gated Recurrent Units to model the temporal sequence of sparse sensor measurements along with a shallow decoder network to reconstruct the full spatio-temporal field from the latent state space. Our algorithm introduces a SINDy-based regularization for which the latent space progressively converges to a SINDy-class functional, provided the projection remains within the set. In restricting SINDy to a linear model, a Koopman-SHRED model is generated. SINDy-SHRED (i) learns a symbolic and interpretable generative model of a parsimonious and low-dimensional latent space for the complex spatio-temporal dynamics, (ii) discovers new physics models even for well-known physical systems, (iii) achieves provably robust convergence with an observed globally convex loss landscape, and (iv) achieves superior accuracy, data efficiency, and training time, all with fewer model parameters. We conduct systematic experimental studies on PDE data such as turbulent flows, real-world sensor measurements for sea surface temperature, and direct video data. The interpretable SINDy and Koopman models of latent state dynamics enable stable and accurate long-term video predictions, outperforming all current baseline deep learning models in accuracy, training time, and data requirements, including Convolutional LSTM, PredRNN, ResNet, and SimVP.

Figures (25)

• Figure 1: (a) Illustration of the SINDy-SHRED and Koopman-SHRED architecture. SINDy-SHRED transfers the original sparse sensor signal (red) to an interpretable latent representation (purple) that falls into the SINDy-class functional. This framework can be adapted into Koopman-SHRED by restricting the library $\Theta(\cdot)$ to be linear. The shallow decoder performs a reconstruction in the state space. We obtain an interpretable linear model for the sea-surface temperature data considered (details in Sec. \ref{['sec:sst_expr']}). (b) The loss function consists of three parts: (i) the SHRED loss controls the reconstruction accuracy of the state space, (ii) the Ensemble SINDy loss helps to model the parsimonious dynamics of the latent space, and (iii) the sparsity constraint identifies the governing equation within this optimization framework. (c) We visualize the globally convex loss landscape of SINDy-SHRED as in li2018visualizing.
• Figure 2: (a) Extrapolation of latent representation in SINDy-SHRED from the discovered dynamical system for flow over a cylinder data. Colored: true latent representation. Grey: SINDy extrapolation. (b) Evolution of dynamical system (\ref{['eqn:flow_equation']}). The left two panels are the phase plane $z_1$ vs $z_2$ and $z_3$ vs $z_4$ respectively. The magenta plus symbols are the fixed points. The nonlinear limit cycle behavior is shown in the right panel for $z_j(t)$. (c) Long-term pixel space video prediction via SINDy-SHRED. We demonstrate the forward prediction outcome up to 180 frames.
• Figure 3: (a) Extrapolation of latent representation in SINDy-SHRED from the discovered dynamical system for the pendulum moving data. Blue: true latent representation. Grey: SINDy extrapolation. (b) The pendulum video generation outcome from ResNet, SimVP, ConvLSTM, PredRNN, and SINDy-SHRED from frame 20 to frame 245.
• Figure 4: (a) Extrapolation of latent representation in SINDy-SHRED from the discovered dynamical system for SST. Colored: true latent representation. Grey: SINDy extrapolation. (b) Decoder reconstruction of three independent directions $z_1, z_2, z_3$ in the latent space. (c) Long-term global sea-surface temperature prediction via SINDy-SHRED from week 0 to week 100. We crop the global map for better visualization.
• Figure 5: Visualizations of latent spaces and predictions for SINDy-SHRED applied to PDE simulations of isotropic turbulence, atmospheric chemistry, and 2D Kolmogorov flow datasets. Left: Extrapolation of latent representations using SINDy-SHRED from the discovered dynamical system for the ozone data, showing the true latent representation (colored lines) and SINDy extrapolation (grey lines). Right: Long-term spatiotemporal predictions from SINDy-SHRED compared to ground truth. The details of the 2D Kolmogorov experiment can be found in the Appendix.

Theorems & Definitions (13)

• Theorem 1
• Theorem 2
• Theorem \ref{thm:xi_error}
• proof
• Theorem \ref{thm:nn_error}
• proof
• Lemma 1: Thm. 2.11 in bartlett2021deep
• Lemma 2: Thm. 2.2 in bartlett2021deep
• Lemma 3
• Lemma 4: Proposition 2 in virmaux2018lipschitz