Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Control of Unknown Linear Switched Systems via Policy Gradient Methods

Felix Laurent, Feiran Zhao, Jaap Eising, Florian Dörfler

TL;DR

This work develops a policy gradient adaptive control (PGAC) framework for stabilizing unknown linear switched systems using online, data-driven gradient updates on the infinite-horizon LQR cost $C(K)$. By employing sliding-window identification of local dynamics and a gradient-descent update, the method tracks switching dynamics and achieves sequential strong stability under a dwell-time condition and bounded variation in system matrices. Theoretical guarantees include bounds on the identification error, the LQR cost, and the state trajectory, with extensions to infinite switches. Simulations corroborate the theory, showing cost upper-bounds and bounded state norms across switches, and illustrating practical stability under changing dynamics.

Abstract

We consider the policy gradient adaptive control (PGAC) framework, which adaptively updates a control policy in real time, by performing data-based gradient descent steps on the linear quadratic regulator cost. This method has empirically shown to react to changing circumstances, such as model parameters, efficiently. To formalize this observation, we design a PGAC method which stabilizes linear switched systems, where both model parameters and switching time are unknown. We use sliding window data for the policy gradient estimate and show that under a dwell time condition and small dynamics variation, the policy can track the switching dynamics and ensure closed-loop stability. We perform simulations to validate our theoretical results.

Adaptive Control of Unknown Linear Switched Systems via Policy Gradient Methods

TL;DR

This work develops a policy gradient adaptive control (PGAC) framework for stabilizing unknown linear switched systems using online, data-driven gradient updates on the infinite-horizon LQR cost . By employing sliding-window identification of local dynamics and a gradient-descent update, the method tracks switching dynamics and achieves sequential strong stability under a dwell-time condition and bounded variation in system matrices. Theoretical guarantees include bounds on the identification error, the LQR cost, and the state trajectory, with extensions to infinite switches. Simulations corroborate the theory, showing cost upper-bounds and bounded state norms across switches, and illustrating practical stability under changing dynamics.

Abstract

We consider the policy gradient adaptive control (PGAC) framework, which adaptively updates a control policy in real time, by performing data-based gradient descent steps on the linear quadratic regulator cost. This method has empirically shown to react to changing circumstances, such as model parameters, efficiently. To formalize this observation, we design a PGAC method which stabilizes linear switched systems, where both model parameters and switching time are unknown. We use sliding window data for the policy gradient estimate and show that under a dwell time condition and small dynamics variation, the policy can track the switching dynamics and ensure closed-loop stability. We perform simulations to validate our theoretical results.
Paper Structure (17 sections, 18 theorems, 59 equations, 2 figures, 2 algorithms)

This paper contains 17 sections, 18 theorems, 59 equations, 2 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Policy gradient methods for the linear quadratic regulator
  4. Policy gradient adaptive control
  5. Policy gradient adaptive control for linear switched systems
  6. Problem formulation and the PGAC method for linear switched systems
  7. Closed-loop stability analysis
  8. Simulations
  9. Upper boundedness of the LQR cost
  10. State norm
  11. Conclusions
  12. Proofs
  13. Proof of Lemma \ref{['error_LS']} (Identification Error)
  14. Proof of upper boundedness of the LQR cost
  15. Proof of Theorem 1 (State Stability)
  16. ...and 2 more sections

Key Result

Lemma 1

For any $K\in\mathcal{S}$, the gradient of $C(K)$ is given by where ${\Sigma_K}$ satisfies Sigma, and $P$ is the positive definite solution to the Lyapunov equation

Figures (2)

  • Figure 1: Comparison between real cost function $C_i(K_t)$ and its upper bound $\overline{C}_i$
  • Figure 2: Comparison between the state $\|x_t\|$ norm and its upper bound.

Theorems & Definitions (27)

  • Lemma 1: fazel2018global
  • Lemma 1: fazel2018global
  • Definition 1: Sequential strong stability
  • Lemma 2
  • Theorem 1: Stability in $N$ switches
  • Corollary 1: Infinite switches
  • Lemma 3
  • Lemma 4: Lemma 11, zhao2025policy
  • Lemma 5: Lemma 12, zhao2025policy
  • Lemma 6
  • ...and 17 more