Information Shapes Koopman Representation

TL;DR

This work reconceptualizes Koopman-based dynamical modeling through the information bottleneck, showing that latent mutual information and von Neumann entropy jointly control temporal coherence and spectral diversity. By deriving an information-theoretic Lagrangian and a practical loss, the authors balance simplicity and expressiveness to avoid mode collapse while preserving relevant Koopman modes near the unit circle. The proposed method yields stable, interpretable latent manifolds and improved forecasting and control across physical, visual, and graph-structured dynamical systems, demonstrating both theoretical and empirical advantages over existing Koopman learning approaches. The framework is architecture-agnostic and supported by public code, enabling broader adoption in diverse dynamical modeling tasks.

Abstract

The Koopman operator provides a powerful framework for modeling dynamical systems and has attracted growing interest from the machine learning community. However, its infinite-dimensional nature makes identifying suitable finite-dimensional subspaces challenging, especially for deep architectures. We argue that these difficulties come from suboptimal representation learning, where latent variables fail to balance expressivity and simplicity. This tension is closely related to the information bottleneck (IB) dilemma: constructing compressed representations that are both compact and predictive. Rethinking Koopman learning through this lens, we demonstrate that latent mutual information promotes simplicity, yet an overemphasis on simplicity may cause latent space to collapse onto a few dominant modes. In contrast, expressiveness is sustained by the von Neumann entropy, which prevents such collapse and encourages mode diversity. This insight leads us to propose an information-theoretic Lagrangian formulation that explicitly balances this tradeoff. Furthermore, we propose a new algorithm based on the Lagrangian formulation that encourages both simplicity and expressiveness, leading to a stable and interpretable Koopman representation. Beyond quantitative evaluations, we further visualize the learned manifolds under our representations, observing empirical results consistent with our theoretical predictions. Finally, we validate our approach across a diverse range of dynamical systems, demonstrating improved performance over existing Koopman learning methods. The implementation is publicly available at https://github.com/Wenxuan52/InformationKoopman.

Figures (15)

• Figure 1: Information-theoretic Koopman framework. (a) Structure overview, (b) Information disentanglement with spectral interpretations, and (c) Water-filling effect of Mutual Information (MI) and von Neumann entropy (VNE) on spectral information allocation.
• Figure 2: Eigenvalue comparison and manifold visualization of the Kármán vortex street. Left: Eigenvalue distributions of Koopman operators. Right: t-SNE visualization of learned latent manifolds for five methods on the Kármán vortex. The underlying dynamics is abstracted as a limit cycle.
• Figure 3: Latent manifolds of Planar visualized using locally linear embedding. The first subfigure shows the ground truth, while the second to fifth depict manifolds learned by different algorithms.
• Figure 4: Comparison of prediction over 100 rollout steps. The left two figures show results for the Rope environment ($n\in [40, 56]$) with and without noise; the right two subfigures are the results for the Soft environment ($n\in [160, 224]$).
• Figure 5: Ablation study on the pendulum task. Latent manifolds are learned from high-dimensional pendulum images, where the ground-truth phase space is isomorphic to $\mathcal{S}^1 \times \mathbb{R}$. Color represents the pendulum angle. Each subplot corresponds to removing or adjusting one regularization term: latent mutual information ($\alpha$), KL divergence ($\beta$), and von Neumann entropy ($\gamma$).

Theorems & Definitions (9)

• Definition 2.1: Koopman Operator koopman1931hamiltonian
• Definition 2.2: Mutual Information mackay2003information
• Definition 2.3: Von Neumann Entropy witten2020mini
• Definition E.1: Density Matrix bach2022information
• Definition E.2: Effective Dimension roy2007effective
• Definition E.3: Entropy, Mutual Information and Conditional Mutual Information
• Lemma F.2: Chain Rule of Information
• Lemma F.3: Pinsker's Inequality yeung2008information
• Remark F.4