Koopman Kernel Regression

TL;DR

Koopman Kernel Regression (KKR) tackles forecasting nonlinear dynamical systems by learning LTI predictors within a Koopman-invariant RKHS, ensuring that learned representations align with linearized dynamics. It constructs eigenfunction-based RKHS spaces and uses a representer-theorem approach to learn an LTI predictor from trajectory data, with a time-discrete kernel formulation that preserves inter-sample dynamics. Theoretical guarantees include universal consistency and a data-dependent generalization bound that scales as $O(H/\, obreak0)$, under mild assumptions, and empirical results show superior forecasting over Koopman operator regression and signature-based methods across several dynamical systems. The framework achieves these guarantees while maintaining computational practicality comparable to existing RKHS-based methods, though it acknowledges limitations related to non-recurrent domains and spectral sampling in complex regimes.

Abstract

Many machine learning approaches for decision making, such as reinforcement learning, rely on simulators or predictive models to forecast the time-evolution of quantities of interest, e.g., the state of an agent or the reward of a policy. Forecasts of such complex phenomena are commonly described by highly nonlinear dynamical systems, making their use in optimization-based decision-making challenging. Koopman operator theory offers a beneficial paradigm for addressing this problem by characterizing forecasts via linear time-invariant (LTI) ODEs, turning multi-step forecasts into sparse matrix multiplication. Though there exists a variety of learning approaches, they usually lack crucial learning-theoretic guarantees, making the behavior of the obtained models with increasing data and dimensionality unclear. We address the aforementioned by deriving a universal Koopman-invariant reproducing kernel Hilbert space (RKHS) that solely spans transformations into LTI dynamical systems. The resulting Koopman Kernel Regression (KKR) framework enables the use of statistical learning tools from function approximation for novel convergence results and generalization error bounds under weaker assumptions than existing work. Our experiments demonstrate superior forecasting performance compared to Koopman operator and sequential data predictors in RKHS.

Figures (12)

• Figure 1: Illustration of on-manifold dynamics of LTI predictors.
• Figure 2: Time complexities.
• Figure 3: Forecasting performance (48 i.i.d. runs) for the bi-stable system for $H{=}14$ and $N{=}50$ for respectively optimal $D_{\text{KKR}}{=}100, D_{\text{PCR}}{=}10$ and 15 delays for Sig-PDEs. Left: Exemplary trajectories showing the advantage of learning with time-series kernels. Right: The generalization gap with an increasing forecast horizon, demonstrating generalization advantages of KKR.
• Figure 4: Forecasting risks (20 i.i.d. runs) for the Van der Pol system over a time-horizon $H=14$ ($T=1s$). Left: Generalization gap for the best $D$ / $l$ (ours 500, PCR 62, RRR 100, RR-Sig-PDE 10) is depicted with a growing number of data points. Right: Test risk behavior with an increasing amount of eigenspaces is shown for $N=200$. Shaded areas depict min-max risk intervals.
• Figure 5: Cumulative error and forecast risks (5 train-test splits) for flow past cylinder data and $H=99$. Our KKR with orders-of-magnitude greater usable $\ell$-range and accuracy. Left: Cumulative absolute error for the best $D/ \ell$ (ours 200/70, PCR 200/35) is depicted over timesteps. Right: Forecast risk for 99 steps within a range of RBF lengthscales. Shaded areas depict min-max intervals.

Theorems & Definitions (27)

• Definition 1
• Lemma 1: Invariance transform
• Theorem 1: Koopman eigenfunction kernel
• Proposition 1: Koopman kernel
• Corollary 1: Time-discrete Koopman kernel
• Proposition 2: KKR
• Remark 1
• Proposition 3
• Theorem 2: Universal consistency
• Theorem 3: Generalization Gap of KKR