A Comparison between Markov Chain and Koopman Operator Based Data-Driven Modeling of Dynamical Systems

TL;DR

The paper addresses data-driven modeling for dynamical systems by contrasting Markov chain-based and Koopman operator-based approaches. It formalizes both frameworks for autonomous dynamics (encoding, linear evolution, calibration, decoding) and extends to controlled systems (MDP-based transitions vs. linear/bilinear lifted dynamics with MPC), using the Van der Pol oscillator as a core benchmark. Through numerical experiments, it demonstrates that Koopman-based models, particularly bilinear variants, generally yield higher predictive accuracy and substantial MPC efficiency, while Markov chain methods offer fast, interpretable control via value iteration; linear Koopman models may falter near nonlinear regimes. The work provides practical guidance on selecting between stochastic Markov formulations and lifted-linear Koopman representations depending on accuracy requirements and computational constraints.

Abstract

Markov chain-based modeling and Koopman operator-based modeling are two popular frameworks for data-driven modeling of dynamical systems. They share notable similarities from a computational and practitioner's perspective, especially for modeling autonomous systems. The first part of this paper aims to elucidate these similarities. For modeling systems with control inputs, the models produced by the two approaches differ. The second part of this paper introduces these models and their corresponding control design methods. We illustrate the two approaches and compare them in terms of model accuracy and computational efficiency for both autonomous and controlled systems in numerical examples.

Figures (5)

• Figure 1: The relation between the spaces for $P$ in Markov chain-based modeling and $A$ in Koopman operator-based modeling.
• Figure 2: Actual trajectory of the unforced Van der Pol oscillator system versus predicted trajectories from Markov chain and Koopman operator based models starting from initial condition $x_0 = (3,3)$.
• Figure 3: Actual trajectory of the forced Van der Pol oscillator system versus predicted trajectories from controlled Markov chain model and Koopman operator-based linear and bilinear models starting from $x_0 = (3,3)$ and under open-loop input signal $u(t) = 2\cos(t)$.
• Figure 4: Closed-loop trajectories under the controllers based on controlled Markov chain model + value iteration, Koopman operator-based linear model + linear MPC, Koopman operator-based bilinear model + linear MPC, and nonlinear model + nonlinear MPC.
• Figure 5: Phase portraits of closed-loop systems under controllers designed from: (a) Controlled Markov chain model + value iteration, (b) Koopman operator-based linear model + linear MPC, (c) Koopman operator-based bilinear model + linear MPC, and (d) Nonlinear model + nonlinear MPC.