Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Koopman Meets Hamilton and Jacobi

Umesh Vaidya

TL;DR

The paper addresses the challenge of solving Hamilton-Jacobi equations in control by linking them to the spectral theory of the Koopman operator. It introduces two constructive procedures that use Koopman eigenfunctions to characterize a Lagrangian submanifold and thus the HJ solution: Procedure One relies on eigenfunctions of the uncontrolled system to create a near-integrable Hamiltonian and solves a Riccati equation to obtain V(x); Procedure Two leverages eigenfunctions of the Hamiltonian system to locate the stable manifold via a finite-dimensional basis, yielding an explicit gradient form for p = ∂V/∂x and an accompanying control law. A convergence analysis for principal eigenfunction approximations is provided, showing consistency as data/basis size grows, and a data-driven path via empirical measures with 1/√L convergence. The framework is demonstrated through two simulations, including an inverted pendulum, and is shown to produce competitive or superior results to LQR and Taylor-series-based controllers, illustrating its potential to extend linear control concepts to nonlinear dynamics.

Abstract

In this paper, we establish a connection between the spectral theory of the Koopman operator and the solution of the Hamilton Jacobi (HJ) equation. The HJ equation occupies a central place in systems theory, and its solution is of interest in various control problems, including optimal control, robust control, and input-output analysis. A Hamiltonian dynamical system can be associated with the HJ equation and the solution of the HJ equation can be extracted from the Hamiltonian system in the form of Lagrangian submanifold. One of the main contributions of this paper is to show that the Lagrangian submanifolds can be obtained using the spectral analysis of the Koopman operator. We present two different procedures for the approximation of the HJ solution. We utilize the spectral properties of the Koopman operator associated with the uncontrolled dynamical system and Hamiltonian systems to approximate the HJ solution. We present a convex optimization-based computational framework with convergence analysis for approximating the Koopman eigenfunctions and the Lagrangian submanifolds. Our solution approach to the HJ equation using Koopman theory provides for a natural extension of results from linear systems to nonlinear systems. We demonstrate the application of this work for solving the optimal control problem. Finally, we present simulation results to validate the paper's main findings and compare them against linear quadratic regulator and Taylor series based approximation controllers.

When Koopman Meets Hamilton and Jacobi

TL;DR

The paper addresses the challenge of solving Hamilton-Jacobi equations in control by linking them to the spectral theory of the Koopman operator. It introduces two constructive procedures that use Koopman eigenfunctions to characterize a Lagrangian submanifold and thus the HJ solution: Procedure One relies on eigenfunctions of the uncontrolled system to create a near-integrable Hamiltonian and solves a Riccati equation to obtain V(x); Procedure Two leverages eigenfunctions of the Hamiltonian system to locate the stable manifold via a finite-dimensional basis, yielding an explicit gradient form for p = ∂V/∂x and an accompanying control law. A convergence analysis for principal eigenfunction approximations is provided, showing consistency as data/basis size grows, and a data-driven path via empirical measures with 1/√L convergence. The framework is demonstrated through two simulations, including an inverted pendulum, and is shown to produce competitive or superior results to LQR and Taylor-series-based controllers, illustrating its potential to extend linear control concepts to nonlinear dynamics.

Abstract

In this paper, we establish a connection between the spectral theory of the Koopman operator and the solution of the Hamilton Jacobi (HJ) equation. The HJ equation occupies a central place in systems theory, and its solution is of interest in various control problems, including optimal control, robust control, and input-output analysis. A Hamiltonian dynamical system can be associated with the HJ equation and the solution of the HJ equation can be extracted from the Hamiltonian system in the form of Lagrangian submanifold. One of the main contributions of this paper is to show that the Lagrangian submanifolds can be obtained using the spectral analysis of the Koopman operator. We present two different procedures for the approximation of the HJ solution. We utilize the spectral properties of the Koopman operator associated with the uncontrolled dynamical system and Hamiltonian systems to approximate the HJ solution. We present a convex optimization-based computational framework with convergence analysis for approximating the Koopman eigenfunctions and the Lagrangian submanifolds. Our solution approach to the HJ equation using Koopman theory provides for a natural extension of results from linear systems to nonlinear systems. We demonstrate the application of this work for solving the optimal control problem. Finally, we present simulation results to validate the paper's main findings and compare them against linear quadratic regulator and Taylor series based approximation controllers.

Paper Structure

This paper contains 13 sections, 9 theorems, 125 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Preliminaries and Notations
  3. Lagrangian Submanifold
  4. Spectral theory of Koopman operator
  5. Main Results
  6. Riccatti Equation
  7. Integrable Hamiltonian System and Koopman Eigenfunctions: Procedure One
  8. Koopman Eigenfunctions of the Hamiltonian System : Procedure Two
  9. Computation of Principal Eigenfunctions: Convergence Analysis
  10. Simulation Results
  11. Example 1
  12. Example 2
  13. Conclusions

Key Result

Proposition 1

Let $S: {\@fontswitch\mathcal{M}}\to {\mathbb R}$ and consider the submanifold ${\@fontswitch\mathcal{L}}_S\subset T^\star{\@fontswitch\mathcal{M}}$ of the form (lag_par). Then if and only if ${\@fontswitch\mathcal{L}}_S$ is an invariant submanifold of the Hamiltonian system (hamiltonsystem).

Figures (8)

  • Figure 1: Principal Eigenfunctions $\phi_1({\mathbf x}),\phi_2({\mathbf x})$ of the uncontrolled system
  • Figure 2: Value function of LQR control and Proc. 1 and Proc.2 (top). Feedback contorol law for LQR, Proc. 1, and Proc.2 (bottom).
  • Figure 3: Principal Eigenfunction error vs data length
  • Figure 4: Comparison of optimal cost function error: $|V^\star({\mathbf x})-V_K({\mathbf x})|$ (left) and $|V^\star({\mathbf x})-V_T({\mathbf x})|$ (right).
  • Figure 5: Comparison of angular position obtained using Procedure one (red), LQR controller (blue), and pole placement controller (yellow) from multiple initial conditions.
  • ...and 3 more figures

Theorems & Definitions (27)

  • Definition 1: Lagrangian submanifold
  • Proposition 1
  • Definition 2: Koopman Operator
  • Definition 3
  • Definition 4: Open Eigenfunction mezic2020spectrum
  • Corollary 1
  • Remark 1
  • Proposition 2
  • proof
  • Proposition 3
  • ...and 17 more