Continuous-time iterative linear-quadratic regulator

TL;DR

The paper introduces a continuous-time equivalent of the iterative linear-quadratic regulator (iLQR) that incorporates a novel regularization scheme grounded in the sufficient conditions of the Riccati pass and a backtracking line-search. The algorithm operates in two passes per iteration—backward to build a local value-function model and forward to update the nominal trajectory—while formulating the problem as a stiff IVP/DAE to leverage adaptive, high-order solvers. The approach is validated on cart-pole swing-up tasks with convex and non-convex costs, demonstrating convergence to locally optimal policies and robust handling of numerical stiffness. The methodology enables efficient, accurate trajectory optimization for continuous-time systems and sets the stage for extensions to constraints and infeasible trajectories in a continuous-time framework.

Abstract

We present a continuous-time equivalent to the well-known iterative linear-quadratic algorithm including an implementation of a backtracking line-search policy and a novel regularization approach based on the necessary conditions in the Riccati pass of the linear-quadratic regulator. This allows the algorithm to effectively solve trajectory optimization problems with non-convex cost functions, which is demonstrated on the cart-pole swing-up problem. The algorithm compatibility with state-of-the-art suites of numerical integration solvers allows for the use of high-order adaptive-step methods. Their use results in a variable number of time steps both between passes of the algorithm and across iterations, maintaining a balance between the number of function evaluations and the discretization error.

Figures (3)

• Figure 1: Diagram of the cart-pole system
• Figure 2: Results for the variant of the problem with convex cost functions.
• Figure 3: Results for the variant of the problem with a non-convex running cost.