Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unifying HJB and Riccati equations: A Koopman operator approach to nonlinear optimal control

Tobias Breiten, Bernhard Höveler

TL;DR

The paper addresses the challenge of connecting the nonlinear HJB framework with a linear-quadratic Riccati structure by embedding nonlinear closed-loop dynamics into an infinite-dimensional Koopman lift and formulating a value bilinear form on a weighted function space. It establishes a spectral, low-rank representation of the value function via $v(z)=\sum_i \sigma_i p_i(z)^2$ and an associated optimal feedback $u_*(z)=\sum_i \sigma_i (b(z)^\top \nabla p_i(z)) p_i(z)$, while proving exponential stability of the Koopman semigroup and deriving a nonlinear operator equation on $\ell_2$ that reduces to the ARE in the linear-quadratic limit. This framework yields a sum-of-squares representation and enables practical low-rank numerical schemes for nonlinear optimal control. A numerical proof-of-concept on a two-dimensional system demonstrates rapid eigenvalue decay and the viability of the approach for constructing high-quality, low-rank approximations of the value function and optimal feedback.

Abstract

This paper proposes an operator-theoretic framework that recasts the minimal value function of a nonlinear optimal control problem as an abstract bilinear form on a suitable function space. The resulting bilinear form is shown to satisfy an operator equation with quadratic nonlinearity obtained by formulating the Lyapunov equation for a Koopman lift of the optimal closed-loop dynamics to an infinite-dimensional state space. It is proven that the minimal value function admits a rapidly convergent sum-of-squares expansion, a direct consequence of the fast spectral decay of the bilinear form. The framework thereby establishes a natural link between the Hamilton-Jacobi-Bellman and a Riccati-like operator equation and further motivates numerical low-rank schemes.

Unifying HJB and Riccati equations: A Koopman operator approach to nonlinear optimal control

TL;DR

The paper addresses the challenge of connecting the nonlinear HJB framework with a linear-quadratic Riccati structure by embedding nonlinear closed-loop dynamics into an infinite-dimensional Koopman lift and formulating a value bilinear form on a weighted function space. It establishes a spectral, low-rank representation of the value function via and an associated optimal feedback , while proving exponential stability of the Koopman semigroup and deriving a nonlinear operator equation on that reduces to the ARE in the linear-quadratic limit. This framework yields a sum-of-squares representation and enables practical low-rank numerical schemes for nonlinear optimal control. A numerical proof-of-concept on a two-dimensional system demonstrates rapid eigenvalue decay and the viability of the approach for constructing high-quality, low-rank approximations of the value function and optimal feedback.

Abstract

This paper proposes an operator-theoretic framework that recasts the minimal value function of a nonlinear optimal control problem as an abstract bilinear form on a suitable function space. The resulting bilinear form is shown to satisfy an operator equation with quadratic nonlinearity obtained by formulating the Lyapunov equation for a Koopman lift of the optimal closed-loop dynamics to an infinite-dimensional state space. It is proven that the minimal value function admits a rapidly convergent sum-of-squares expansion, a direct consequence of the fast spectral decay of the bilinear form. The framework thereby establishes a natural link between the Hamilton-Jacobi-Bellman and a Riccati-like operator equation and further motivates numerical low-rank schemes.

Paper Structure

This paper contains 10 sections, 22 theorems, 172 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Existing literature and related results
  3. Contribution
  4. Notation
  5. Koopman and Perron-Frobenius semigroup
  6. Weighted Lipschitz spaces and their predual
  7. The Koopman semigroup on $\mathrm{W}^{1,\infty}_{ w }\left( \Omega \right)$
  8. The value function as a bilinear form
  9. Numerical proof of concept
  10. Technical proofs

Key Result

Lemma 2.2

Let $1 \le p < \infty$ and $p^*$ such that $\frac{1}{p} + \frac{1}{p^\ast} = 1$. Then $\mathrm{L}_{w}^{ p^* }\left( \Omega \right) \simeq \left(\mathrm{L}_{w}^{ p }\left( \Omega \right)\right)^*$, with the dual pairing: where $\phi \in \mathrm{L}_{w}^{ p }\left( \Omega \right)$ and $\varphi \in \mathrm{L}_{w}^{ p^* }\left( \Omega \right)$.

Figures (1)

  • Figure 1: Left: Computed value function $v$ for the modified Van-der-Pol oscillator. Right: The first 60 eigenvalues $\sigma_i$ for the eigendecomposition of the value bilinear form.

Theorems & Definitions (68)

  • Definition 2.1
  • Lemma 2.2
  • Definition 2.3
  • Remark 2.4
  • Remark 2.5
  • Lemma 2.6
  • Proof 1
  • Definition 2.7
  • Remark 2.8
  • Remark 2.9
  • ...and 58 more