Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Regret of Recursive Methods for Discrete-Time Adaptive Control with Matched Uncertainty

Aren Karapetyan, Efe C. Balta, Anastasios Tsiamis, Andrea Iannelli, John Lygeros

TL;DR

This work addresses finite-regret adaptive control for discrete-time nonlinear systems with matched uncertainty by formulating online parameter learning as minimization of a cost that aligns with uncertainty cancellation. It introduces Recursive Proximal Learning (RPL), a proximal-point-based update, and proves parameter-space contraction and closed-loop finite regret under a sufficient excitation condition, with improved constants under additional assumptions. It also analyzes Recursive Least Squares with Exponential Forgetting (RLSFF), showing finite regret under a stronger persistence of excitation condition and comparing the two methods in terms of regret performance. A discrete-time MRAC example demonstrates that RPL can achieve superior tracking and regret performance compared with RLSFF and a reference governor MRAC, highlighting the practical viability of causal, online, uncertainty-matching adaptive controllers. The results provide a principled path to achieving both stability and performance guarantees in adaptive control under time-varying dynamics, with clear guidance on excitation requirements and method selection for practitioners.

Abstract

Continuous-time adaptive controllers for systems with a matched uncertainty often comprise an online parameter estimator and a corresponding parameterized controller to cancel the uncertainty. However, such methods are often impossible to implement directly, as they depend on an unobserved estimation error. We consider the equivalent discrete-time setting with a causal information structure, and propose a novel, online proximal point method-based adaptive controller, that under a sufficient excitation (SE) condition is asymptotically stable and achieves finite regret, scaling only with the time required to fulfill the SE. We show the same also for the widely-used recursive least squares with exponential forgetting controller under a stronger persistence of excitation condition.

On the Regret of Recursive Methods for Discrete-Time Adaptive Control with Matched Uncertainty

TL;DR

This work addresses finite-regret adaptive control for discrete-time nonlinear systems with matched uncertainty by formulating online parameter learning as minimization of a cost that aligns with uncertainty cancellation. It introduces Recursive Proximal Learning (RPL), a proximal-point-based update, and proves parameter-space contraction and closed-loop finite regret under a sufficient excitation condition, with improved constants under additional assumptions. It also analyzes Recursive Least Squares with Exponential Forgetting (RLSFF), showing finite regret under a stronger persistence of excitation condition and comparing the two methods in terms of regret performance. A discrete-time MRAC example demonstrates that RPL can achieve superior tracking and regret performance compared with RLSFF and a reference governor MRAC, highlighting the practical viability of causal, online, uncertainty-matching adaptive controllers. The results provide a principled path to achieving both stability and performance guarantees in adaptive control under time-varying dynamics, with clear guidance on excitation requirements and method selection for practitioners.

Abstract

Continuous-time adaptive controllers for systems with a matched uncertainty often comprise an online parameter estimator and a corresponding parameterized controller to cancel the uncertainty. However, such methods are often impossible to implement directly, as they depend on an unobserved estimation error. We consider the equivalent discrete-time setting with a causal information structure, and propose a novel, online proximal point method-based adaptive controller, that under a sufficient excitation (SE) condition is asymptotically stable and achieves finite regret, scaling only with the time required to fulfill the SE. We show the same also for the widely-used recursive least squares with exponential forgetting controller under a stronger persistence of excitation condition.
Paper Structure (11 sections, 5 theorems, 50 equations, 1 figure)

This paper contains 11 sections, 5 theorems, 50 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Online Estimation
  4. Regret
  5. Recursive Proximal Learning
  6. Contraction in Parameter Space
  7. Closed-loop Analysis
  8. RLS with Exponential Forgetting
  9. Analysis of the Bounds
  10. Numerical Simulation
  11. Conclusion

Key Result

Lemma 1

Assume $B_k \phi_k^\top$ is sufficiently exciting, then the RPL estimate eq:proximal_update satisfies where $\eta = \frac{\varepsilon}{\delta+\varepsilon} \in (0,1)$. Moreover, if Assumption assum:beta_bound holds, and $\varepsilon < \frac{\delta\sqrt{\delta}}{\sqrt{\beta}-\sqrt{\delta}}$ then there exists a $\gamma \in (0,1)$ such that

Figures (1)

  • Figure 1: Comparison of adaptive controllers for the MRAC example in terms of asymptotic tracking (on the top) and regret (on the bottom).

Theorems & Definitions (11)

  • Definition 1
  • Lemma 1
  • proof
  • Theorem 1
  • proof
  • Lemma 2
  • proof
  • Definition 2
  • Lemma 3
  • Theorem 2
  • ...and 1 more