Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Kernel-Based LMI Approaches to Solving the Hamilton-Jacobi-Bellman Equation and Nonlinear Optimal Control

Boumediene Hamzi, Umesh Vaidya

TL;DR

A kernel-based linear matrix inequality (LMI) approach for solving Hamilton-Jacobi-Bellman equations arising in nonlinear optimal control, with comprehensive validation through multiple initial conditions showing that all closed-loop trajectories converge exponentially to the origin from various starting points.

Abstract

We present a kernel-based linear matrix inequality (LMI) approach for solving Hamilton-Jacobi-Bellman (HJB) equations arising in nonlinear optimal control. The method converts the nonlinear HJB inequality into a convex semidefinite program through reproducing kernel Hilbert space (RKHS) representations and Schur complement reformulations. We provide complete theoretical results including convex reformulation, RKHS approximation, finite-dimensional discretization, suboptimality bounds, stability guarantees, and convergence rates. A critical component of our approach is the use of a Riccati Hessian constraint at the equilibrium to prevent trivial solutions while ensuring consistency with linearized optimal control theory. Numerical results on both 1D and 2D systems demonstrate the effectiveness of the approach, with comprehensive validation through multiple initial conditions showing that all closed-loop trajectories converge exponentially to the origin from various starting points, successfully stabilizing unstable equilibria with theoretical guarantees. The method maintains computational tractability while providing rigorous optimality and stability guarantees.

Kernel-Based LMI Approaches to Solving the Hamilton-Jacobi-Bellman Equation and Nonlinear Optimal Control

TL;DR

A kernel-based linear matrix inequality (LMI) approach for solving Hamilton-Jacobi-Bellman equations arising in nonlinear optimal control, with comprehensive validation through multiple initial conditions showing that all closed-loop trajectories converge exponentially to the origin from various starting points.

Abstract

We present a kernel-based linear matrix inequality (LMI) approach for solving Hamilton-Jacobi-Bellman (HJB) equations arising in nonlinear optimal control. The method converts the nonlinear HJB inequality into a convex semidefinite program through reproducing kernel Hilbert space (RKHS) representations and Schur complement reformulations. We provide complete theoretical results including convex reformulation, RKHS approximation, finite-dimensional discretization, suboptimality bounds, stability guarantees, and convergence rates. A critical component of our approach is the use of a Riccati Hessian constraint at the equilibrium to prevent trivial solutions while ensuring consistency with linearized optimal control theory. Numerical results on both 1D and 2D systems demonstrate the effectiveness of the approach, with comprehensive validation through multiple initial conditions showing that all closed-loop trajectories converge exponentially to the origin from various starting points, successfully stabilizing unstable equilibria with theoretical guarantees. The method maintains computational tractability while providing rigorous optimality and stability guarantees.
Paper Structure (16 sections, 11 theorems, 92 equations, 3 figures, 4 tables)

This paper contains 16 sections, 11 theorems, 92 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Theoretical Framework
  4. Preliminary: Schur Complement
  5. HJB Inequality as Linear Matrix Inequality
  6. RKHS Representation of the Value Function
  7. Avoiding Trivial Solutions: The Riccati Hessian Constraint
  8. Finite-Dimensional Collocation System with Riccati Constraint
  9. Performance Guarantees
  10. Relationship to Classical Lyapunov Function Construction
  11. Numerical Results
  12. 1D Scalar Polynomial System
  13. 2D Radially Symmetric System
  14. Numerical Experiment III: Van der Pol Oscillator
  15. Summary and Comparison
  16. ...and 1 more sections

Key Result

Lemma 3.1

Let $M$ be a symmetric block matrix of the form where $A \in \mathbb{R}^{p \times p}$, $B \in \mathbb{R}^{p \times q}$, and $C \in \mathbb{R}^{q \times q}$ with $C \succ 0$ (positive definite). Then More generally, if $C \succeq 0$, then $M \succeq 0$ if and only if $A \succeq 0$, $\text{range}(B) \subseteq \text{range}(C)$, and $A - BC^\dagger B^\top \succeq 0$, where $C^\dagger$ denotes the Mo

Figures (3)

  • Figure 1: Comprehensive numerical results for the 1D scalar system with multiple initial conditions. (a) Value function comparison showing true optimal $V^*(x)$ and kernel approximation $\hat{V}(x)$, with 25 kernel centers marked. (b) Control law comparison between optimal $u^*(x)$ and kernel approximation. (c) Approximation errors on logarithmic scale. (d) State trajectories from six initial conditions $x_0 \in \{-1.2, -0.8, -0.4, 0.4, 0.8, 1.2\}$, all converging to the origin. (e) Convergence analysis on logarithmic scale showing exponential decay of $|x(t)|$ for all trajectories. (f) Summary statistics confirming all final states reach $|x(10)| < 10^{-6}$. The Riccati Hessian constraint ensures exponential convergence from all tested initial conditions.
  • Figure 2: Comprehensive numerical results for the 2D radially symmetric system with multiple initial conditions. (a) True optimal value function $V^*(x)$ with 100 kernel centers. (b) Kernel approximation exhibiting similar radial structure. (c) Approximation error with largest deviations near boundaries. (d) Phase plane showing eight trajectories starting from uniformly distributed points on a unit circle, all spiraling inward to the origin. Initial conditions marked with circles, final states with crosses. (e) Convergence analysis on logarithmic scale showing exponential decay of $\|x(t)\|$ for all eight trajectories. (f) Summary statistics confirming all trajectories converge with final norms $\|x(10)\| < 10^{-2}$. The Riccati Hessian constraint ensures successful stabilization from all directions.
  • Figure 3: Comprehensive numerical results for the Van der Pol oscillator ($\mu=1.0$) using the kernel-based LMI/SDP approach. (a) Quadratic LQR reference $V_{\mathrm{quad}}(x)$, (b) kernel SDP approximation $\hat{V}(x)$, (c) absolute error $|\hat{V} - V_{\mathrm{quad}}|$, (d) phase--plane trajectories from eight initial conditions, (e) exponential convergence to the origin, (f) convergence summary. All trajectories converge exponentially, validating the effectiveness of the Riccati--Hessian constraint.

Theorems & Definitions (31)

  • Lemma 3.1: Schur Complement
  • proof
  • Theorem 3.2: HJB Inequality as LMI
  • proof
  • Remark 3.3: Significance of LMI Reformulation
  • Definition 3.4: Reproducing Kernel Hilbert Space
  • Theorem 3.5: RKHS Approximation
  • proof
  • Remark 3.6: Approximation Quality
  • Definition 3.7: Equilibrium Constraints
  • ...and 21 more