Koopman Operators in Robot Learning

TL;DR

The paper addresses runtime learning in robotics by advocating Koopman operator theory, which linearizes nonlinear dynamics in a lifted observable space and supports online updates with limited data. It presents foundational concepts, data-driven estimation methods like EDMD and HVOK, and strategies for incorporating inputs, followed by a detailed mapping to robotics tasks including control, estimation, and planning. It surveys implementations across manipulation, ground, soft, aerial, and multi-agent systems, and discusses advanced topics such as continuous-time formulations and invariant-subspace mining, while outlining robustness and open challenges. The practical impact lies in enabling data-efficient, interpretable, and compute-friendly models that integrate with Linear Quadratic and Model Predictive Control tools, offering a scalable path toward real-time adaptive robotics. The review also points to hands-on tutorials and code resources to facilitate adoption and experimentation in real-world settings.

Abstract

Koopman operator theory offers a rigorous treatment of dynamics and has been emerging as an alternative modeling and learning-based control method across various robotics sub-domains. Due to its ability to represent nonlinear dynamics as a linear (but higher-dimensional) operator, Koopman theory offers a fresh lens through which to understand and tackle the modeling and control of complex robotic systems. Moreover, it enables incremental updates and is computationally inexpensive, thus making it particularly appealing for real-time applications and online active learning. This review delves deeply into the foundations of Koopman operator theory and systematically builds a bridge from theoretical principles to practical robotic applications. We begin by explaining the mathematical underpinnings of the Koopman framework and discussing approximation approaches for incorporating inputs into Koopman-based modeling. Foundational considerations, such as data collection strategies as well as the design of lifting functions for effective system embedding, are also discussed. We then explore how Koopman-based models serve as a unifying tool for a range of robotics tasks, including model-based control, real-time state estimation, and motion planning. The review proceeds to a survey of cutting-edge research that demonstrates the versatility and growing impact of Koopman methods across diverse robotics sub-domains: from aerial and legged platforms to manipulators, soft-bodied systems, and multi-agent networks. A presentation of more advanced theoretical topics, necessary to push forward the overall framework, is included. Finally, we reflect on some key open challenges that remain and articulate future research directions that will shape the next phase of Koopman-inspired robotics. To support practical adoption, we provide a hands-on tutorial with executable code at https://shorturl.at/ouE59.

Figures (6)

• Figure 1: Overview of the utility and integration of Koopman operator theory and practice in robotics.
• Figure 2: Conceptual illustration of Koopman Operator Theory. The nonlinear dynamics in the original state space $x_t \in \mathcal{X}$, governed by an unknown map $T$, are lifted via observable functions $g(x)$ into a higher-dimensional function space known as the Koopman space. In this lifted domain, the evolution of the system becomes linear and is governed by the Koopman operator $\mathcal{K}$, such that $g(x_{t+1}) = [\mathcal{K}g](x_t)$. Though $T$ and $\mathcal{K}$ act on different spaces, they evolve the same dynamics.
• Figure 3: Left: EDMD's residual error for linear system $x^+ = 0.6 x$ and the family of dictionaries $D_{\beta}(x) = [x, x + \beta \sin(x)]$ with $\beta \in [0.01, 100]$. Note that for all $\beta \in \mathbb{R} \setminus \{0\}$, all dictionaries span the same subspace $\mathcal{S} = \operatorname{span}\{x,\sin(x)\}$. The residual error depends on the choice of basis for subspace $\mathcal{S}$. More importantly, $\mathcal{S}$ is not Koopman-invariant but the residual error can be arbitrarily close to zero depending on the basis. Right: The square root of consistency index for the system and family of dictionaries. Unlike EDMD's residual error, the consistency index does not depend on the choice of basis and accurately measures the approximation quality of the subspace.
• Figure 4: We consider here the pendulum $[\dot{\theta}, \dot{\omega}] = [\omega, -9.81 \sin(\theta) - 0.1\omega]$ and a parametric family of dictionaries comprised of 5 functions in the form $\mathbf{\Psi}(\theta,\omega) = [\theta, \omega, \text{NN}_1, \text{NN}_2,\text{NN}_3]$, where each NN is a feedforward neural network. The plot compares the prediction of the pendulum's angle evolution given linear predictors on the subspaces learned by minimizing the consistency index (equivalent to robust optimization \ref{['eq:robust-learning']}) and minimizing the residual error of EDMD (cf. \ref{['eq:learning-loss-optim-reformulation']}). The subspace learned by minimizing the consistency index is superior in long-term prediction. This is due to the fact that it accounts for all (uncountably many) members of the function space rather than only finitely many members considered in the residual error of EDMD.
• Figure 5: The variable $\epsilon \in [0,1]$ in T-SSD sets the balance between the accuracy and expressiveness of the model. Both SSD and EDMD algorithms are special cases of T-SSD with $\epsilon =0$ and $\epsilon =1$, respectively. (Image taken from MH-JC:23-auto and is available under license (CC BY 4.0): https://creativecommons.org/licenses/by/4.0.)

Theorems & Definitions (1)

• Remark 2.1