Closed Form HJB Solution for Continuous-Time Optimal Control of a Non-Linear Input-Affine System
Akash Vyas, Shreyas Kumar, Jayant Kumar Mohanta, Ravi Prakash
TL;DR
The paper tackles the long-standing challenge of solving the Hamilton-Jacobi-Bellman equation for nonlinear continuous-time systems by deriving a closed-form analytical solution for a class of known-dynamics, input-affine models. By reformulating the system with an augmented drift-free form and leveraging Lyapunov stability, it yields a direct, non-iterative control law that guarantees asymptotic convergence rather than mere boundedness. The authors apply the approach to both set-point regulation and optimal tracking, and extend it to a differential game with disturbances, demonstrating improved performance metrics such as ITSE, cumulative cost, and computation time relative to adaptive, learning-based methods. The work offers a computationally efficient alternative to reinforcement learning/ADP for safety-critical applications, providing explicit guarantees and broad numerical validation on benchmark problems.
Abstract
Designing optimal controllers for nonlinear dynamical systems often relies on reinforcement learning and adaptive dynamic programming (ADP) to approximate solutions of the Hamilton Jacobi Bellman (HJB) equation. However, these methods require iterative training and depend on an initially admissible policy. This work introduces a new analytical framework that yields closed-form solutions to the HJB equation for a class of continuous-time nonlinear input-affine systems with known dynamics. Unlike ADP-based approaches, it avoids iterative learning and numerical approximation. Lyapunov theory is used to prove the asymptotic stability of the resulting closed-loop system, and theoretical guarantees are provided. The method offers a closed-form control policy derived from the HJB framework, demonstrating improved computational efficiency and optimal performance on state-of-the-art optimal control problems in the literature.