Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bayesian Transfer Operators in Reproducing Kernel Hilbert Spaces

Septimus Boshoff, Sebastian Peitz, Stefan Klus

TL;DR

This work tackles scalability and hyperparameter tuning in kernel-based Koopman/DMD methods by casting the embedded Perron–Frobenius operator as a random object in a reproducing kernel Hilbert space and inferring it with sparse variational Gaussian processes. It unifies GP regression with DMD to yield a probabilistic kernel DMD (GP-DMD) that provides uncertainty quantification for eigenfunctions and multi-step forecasts, while allowing sparse dictionary learning and automatic hyperparameter selection. The method recovers the standard EDMD update as a special case with a Tikhonov regularization and extends it with a principled Bayesian training pipeline, including a variational objective (VFE) and active-pseudo-input selection. Numerical experiments on nonlinear oscillators and stochastic wells demonstrate improved generalization under noise, effective reprojection for long-horizon forecasts, and interpretable eigenfunction uncertainty. The results highlight the practical value of combining Bayesian and frequentist viewpoints for kernel transfer operators and point to extensions to heteroskedastic and input-noise settings.

Abstract

The Koopman operator, as a linear representation of a nonlinear dynamical system, has been attracting attention in many fields of science. Recently, Koopman operator theory has been combined with another concept that is popular in data science: reproducing kernel Hilbert spaces. We follow this thread into Gaussian process methods, and illustrate how these methods can alleviate two pervasive problems with kernel-based Koopman algorithms. The first being sparsity: most kernel methods do not scale well and require an approximation to become practical. We show that not only can the computational demands be reduced, but also demonstrate improved resilience against sensor noise. The second problem involves hyperparameter optimization and dictionary learning to adapt the model to the dynamical system. In summary, the main contribution of this work is the unification of Gaussian process regression and dynamic mode decomposition.

Bayesian Transfer Operators in Reproducing Kernel Hilbert Spaces

TL;DR

This work tackles scalability and hyperparameter tuning in kernel-based Koopman/DMD methods by casting the embedded Perron–Frobenius operator as a random object in a reproducing kernel Hilbert space and inferring it with sparse variational Gaussian processes. It unifies GP regression with DMD to yield a probabilistic kernel DMD (GP-DMD) that provides uncertainty quantification for eigenfunctions and multi-step forecasts, while allowing sparse dictionary learning and automatic hyperparameter selection. The method recovers the standard EDMD update as a special case with a Tikhonov regularization and extends it with a principled Bayesian training pipeline, including a variational objective (VFE) and active-pseudo-input selection. Numerical experiments on nonlinear oscillators and stochastic wells demonstrate improved generalization under noise, effective reprojection for long-horizon forecasts, and interpretable eigenfunction uncertainty. The results highlight the practical value of combining Bayesian and frequentist viewpoints for kernel transfer operators and point to extensions to heteroskedastic and input-noise settings.

Abstract

The Koopman operator, as a linear representation of a nonlinear dynamical system, has been attracting attention in many fields of science. Recently, Koopman operator theory has been combined with another concept that is popular in data science: reproducing kernel Hilbert spaces. We follow this thread into Gaussian process methods, and illustrate how these methods can alleviate two pervasive problems with kernel-based Koopman algorithms. The first being sparsity: most kernel methods do not scale well and require an approximation to become practical. We show that not only can the computational demands be reduced, but also demonstrate improved resilience against sensor noise. The second problem involves hyperparameter optimization and dictionary learning to adapt the model to the dynamical system. In summary, the main contribution of this work is the unification of Gaussian process regression and dynamic mode decomposition.

Paper Structure

This paper contains 31 sections, 1 theorem, 65 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Dynamic Mode Decomposition
  3. Gaussian Process Regression
  4. Contributions of this work
  5. Gaussian Process Dynamic Mode Decomposition
  6. Operators as Random Variables
  7. Sparse Variational Bayesian Transfer Operators
  8. Koopman Mode Decomposition
  9. Prediction Theory
  10. Model Selection
  11. Numerical Experiments
  12. Generalization Errors
  13. Decoupling the Lifted Observation Model
  14. Reprojections
  15. Eigenfunction Uncertainty Quantification
  16. ...and 16 more sections

Key Result

Proposition C.3

If $\, \mathbb{E}_{Y \mid X}\left[g(Y) \! \mid X = \! \cdot \right] \! \in \! \mathbb{H}_X \; \forall \; g \in \mathbb{H}_Y$, then $\mathcal{C}_{XX} \mathbb{E}_{Y \mid X} \left[g(Y) \mid X \! = \! \cdot \right] = \mathcal{C}_{XY}g$, see fukumizu2004dimensionality.

Figures (6)

  • Figure 1: Multi-step predictions (solid lines) with $95.45\%$ confidence intervals (shaded regions), and reference values (dashed lines), c.f. Figure \ref{['fig: vdP propagate reprojections']}.
  • Figure 2: Generalization errors. Comparing SMAPE over multi-step forecasts of GP-TCCA (blue), Exact DMD (orange), and a variational sparse GP (green), with hyperparameters found via VFE (dashed) and VAMP-$2$ (solid), c.f. Figure \ref{['fig: decoupled']}.
  • Figure 3: Generalization errors of the decoupled model. Blue: $\sigma_\mathcal{Y} = \sigma_Y$, purple: $\sigma_\mathcal{Y} \neq \sigma_Y$, red: $\sigma_\mathcal{Y} = 0$, c.f. Figure \ref{['fig: combined_SMAPE_vdP']}.
  • Figure 4: Multi-step predictions (solid lines) with reprojections (diamond markers), c.f. Figure \ref{['fig: vdP propagate uncertainty']}.
  • Figure 5: The $2^\text{nd}$ eigenfunction of the stochastic double-well. The coloring illustrates the $68.27\%$ confidence intervals.
  • ...and 1 more figures

Theorems & Definitions (8)

  • Definition 2.1
  • Definition A.1
  • Definition B.1
  • Definition B.2
  • Definition C.1
  • Definition C.2
  • Proposition C.3
  • Definition C.4