Stability of Certainty-Equivalent Adaptive LQR for Linear Systems with Unknown Time-Varying Parameters

TL;DR

The paper tackles online stabilization of discrete-time linear time-varying systems with unknown time-varying parameters and bounded disturbances. It introduces a modular pipeline that combines a projected least-mean-square (LMS) model learner with a certainty-equivalent linear-quadratic regulator (LQR) controller, enabling non-episodic updates at every time step. A time-varying Lyapunov analysis shows finite-gain $\ell^2$-stability of the closed-loop interconnection under a small parameter-diameter and step size, even without persistent excitation, and simulations on a nonlinear planar quadrotor demonstrate practical stability and potential parameter learning. The method is computationally light, robust to parameter drift, and suitable for real-time online adaptation, with future work aimed at proving parameter convergence under excitation and extending to broader system classes.

Abstract

Standard model-based control design deteriorates when the system dynamics change during operation. To overcome this challenge, online and adaptive methods have been proposed in the literature. In this work, we consider the class of discrete-time linear systems with unknown time-varying parameters. We propose a simple, modular, and computationally tractable approach by combining two classical and well-known building blocks from estimation and control: the least mean square filter and the certainty-equivalent linear quadratic regulator. Despite both building blocks being simple and off-the-shelf, our analysis shows that they can be seamlessly combined to a powerful pipeline with stability guarantees. Namely, finite-gain $\ell^2$-stability of the closed-loop interconnection of the unknown system, the parameter estimator, and the controller is proven, despite the presence of unknown disturbances and time-varying parametric uncertainties. Real-world applicability of the proposed algorithm is showcased by simulations carried out on a nonlinear planar quadrotor.

Figures (3)

• Figure 1: The interconnection of the unknown system, the parameter estimator, and the controller.
• Figure 2: Closed-loop position trajectories of the proposed adaptive controller (Alg. \ref{['alg:LMS+LQR']}) (solid) and a non-adaptive baseline (diverging dashed, see zoomed inset on the right) applied to the nonlinear planar quadrotor \ref{['eq:drone_nonlinear']} for Cases (a) and (b).
• Figure 3: Comparison of the proposed adaptive controller (Alg. \ref{['alg:LMS+LQR']}) with (dotted) and without (dashed) additional exploratory noise $\epsilon_k$ in terms of convergence of the estimates to the true values (solid) on the linearized dynamics for Cases (a) and (b).

Theorems & Definitions (12)

• definition 1
• proposition 1
• lemma 1
• proof
• proposition 2
• proof
• theorem 1
• proof
• lemma 2
• proof