Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Parametric SVD of Koopman Operator for Stochastic Dynamical Systems

Minchan Jeong, J. Jon Ryu, Se-Young Yun, Gregory W. Wornell

TL;DR

This work tackles learning the top-$k$ singular subspaces of the Koopman operator for stochastic dynamical systems without relying on numerically unstable SVDs or matrix inversions during training. By formulating a low-rank approximation objective, $\mathcal{L}_{\mathsf{lora}}(\mathbf{f},\mathbf{g}) = -2\mathsf{tr}(\mathsf{T}[\mathbf{f},\mathbf{g}]) + \mathsf{tr}(\mathsf{M}_{\rho_0}[\mathbf{f}]\mathsf{M}_{\rho_1}[\mathbf{g}])$, the method directly minimizes the Hilbert–Schmidt error between the true Koopman operator and its rank-$k$ surrogate, with unbiased gradients due to its polynomial dependence on second-moment matrices. It also leverages nesting (joint or sequential) to improve convergence and stability, and proposes two practical inference routes: (i) a CCA+LoRA approach for exact ordered singular functions and Koopman matrices, and (ii) an EDMD-based approach projecting onto a learned basis for forward/backward predictions. Across synthetic and real-world benchmarks, LoRA variants outperform VAMPnet and DPNet baselines in learning dominant subspaces, eigenfunctions, and long-horizon dynamics, while providing superior stability and scalability. The results suggest that LoRA offers a practical, theoretically grounded path to robust, data-driven Koopman analysis in stochastic systems with broad applicability to molecular dynamics and related fields.

Abstract

The Koopman operator provides a principled framework for analyzing nonlinear dynamical systems through linear operator theory. Recent advances in dynamic mode decomposition (DMD) have shown that trajectory data can be used to identify dominant modes of a system in a data-driven manner. Building on this idea, deep learning methods such as VAMPnet and DPNet have been proposed to learn the leading singular subspaces of the Koopman operator. However, these methods require backpropagation through potentially numerically unstable operations on empirical second moment matrices, such as singular value decomposition and matrix inversion, during objective computation, which can introduce biased gradient estimates and hinder scalability to large systems. In this work, we propose a scalable and conceptually simple method for learning the top-$k$ singular functions of the Koopman operator for stochastic dynamical systems based on the idea of low-rank approximation. Our approach eliminates the need for unstable linear-algebraic operations and integrates easily into modern deep learning pipelines. Empirical results demonstrate that the learned singular subspaces are both reliable and effective for downstream tasks such as eigen-analysis and multi-step prediction.

Efficient Parametric SVD of Koopman Operator for Stochastic Dynamical Systems

TL;DR

This work tackles learning the top- singular subspaces of the Koopman operator for stochastic dynamical systems without relying on numerically unstable SVDs or matrix inversions during training. By formulating a low-rank approximation objective, , the method directly minimizes the Hilbert–Schmidt error between the true Koopman operator and its rank- surrogate, with unbiased gradients due to its polynomial dependence on second-moment matrices. It also leverages nesting (joint or sequential) to improve convergence and stability, and proposes two practical inference routes: (i) a CCA+LoRA approach for exact ordered singular functions and Koopman matrices, and (ii) an EDMD-based approach projecting onto a learned basis for forward/backward predictions. Across synthetic and real-world benchmarks, LoRA variants outperform VAMPnet and DPNet baselines in learning dominant subspaces, eigenfunctions, and long-horizon dynamics, while providing superior stability and scalability. The results suggest that LoRA offers a practical, theoretically grounded path to robust, data-driven Koopman analysis in stochastic systems with broad applicability to molecular dynamics and related fields.

Abstract

The Koopman operator provides a principled framework for analyzing nonlinear dynamical systems through linear operator theory. Recent advances in dynamic mode decomposition (DMD) have shown that trajectory data can be used to identify dominant modes of a system in a data-driven manner. Building on this idea, deep learning methods such as VAMPnet and DPNet have been proposed to learn the leading singular subspaces of the Koopman operator. However, these methods require backpropagation through potentially numerically unstable operations on empirical second moment matrices, such as singular value decomposition and matrix inversion, during objective computation, which can introduce biased gradient estimates and hinder scalability to large systems. In this work, we propose a scalable and conceptually simple method for learning the top- singular functions of the Koopman operator for stochastic dynamical systems based on the idea of low-rank approximation. Our approach eliminates the need for unstable linear-algebraic operations and integrates easily into modern deep learning pipelines. Empirical results demonstrate that the learned singular subspaces are both reliable and effective for downstream tasks such as eigen-analysis and multi-step prediction.

Paper Structure

This paper contains 48 sections, 10 theorems, 60 equations, 7 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Problem Setting and Preliminaries
  3. Discrete-Time Dynamical Systems
  4. Continuous-Time Dynamical Systems
  5. Data-Driven Learning of Singular Subspaces: VAMPnet and DPNet
  6. Common Setup
  7. VAMPnet
  8. DPNet
  9. Practical Limitations of VAMPnet and DPNet
  10. Proposed Methods
  11. Learning
  12. Inference
  13. Approach 1: CCA + LoRA
  14. Approach 2: Extended DMD
  15. Experiments
  16. ...and 33 more sections

Key Result

Proposition 3.1

Let $\mathpzc{K}\colon L_{\rho_1}^2(\mathcal{X})\to L_{\rho_0}^2(\mathcal{X})$ be a compact operator having SVD $\sum_{i=1}^{\infty} \sigma_i \phi_i\otimes\psi_i$ with $\sigma_1\ge\sigma_2\ge \ldots\ge 0$. Let $(\mathbf{f}^\star,\mathbf{g}^\star)$ denote a global minimizer of $\mathcal{L}_{\mathsf{l

Figures (7)

  • Figure 1: Summary of the ordered MNIST experiment. The shaded area indicates $\pm 1$ standard deviation.
  • Figure 2: Visualization of the eigenfunctions of the 1D Langevin dynamics, learned by LoRA$_{\mathsf{seq}}$. In the first panel, the learned eigenfunctions across training iterations are overlaid, with later iterations displayed with lower transparency (red). The dashed lines indicate the true eigenfunctions.
  • Figure 2: Test VAMP-2 scores on the 300 K chignolin dataset marshall2024. Algorithms marked with $\dagger$ diverged during training and were therefore omitted. Scores are the mean of five runs with their standard deviation.
  • Figure 3: Estimated timescales for chignolin's eigenmodes. Note that LoRA$_{\mathsf{jnt}}$ predicts the longest timescale.
  • Figure 4: Summary of the noisy logistic map experiment. The first row reports relative error in singular values, and the second row the directed Hausdorff distance of the estimated eigenvalues to the three most dominant underlying eigenvalues. The shaded area indicates $\pm 0.5$ standard deviation.
  • ...and 2 more figures

Theorems & Definitions (15)

  • Proposition 3.1: Optimality of LoRA loss; see, e.g., Ryu--Xu--Erol--Bu--Zheng--Wornell2024
  • Theorem B.1
  • Theorem B.2: Consistency of DPNet objectives
  • Proposition C.1
  • proof
  • Theorem C.1
  • Lemma C.1: Bernstein's inequality Tropp2015
  • Lemma C.2: Matrix Bernstein inequality with intrinsic dimension Minsker2017, Tropp2015
  • proof : Proof of Theorem \ref{['thm:learning']}
  • Theorem D.1
  • ...and 5 more