Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Closed-Loop Finite-Time Analysis of Suboptimal Online Control

Aren Karapetyan, Efe C. Balta, Andrea Iannelli, John Lygeros

TL;DR

This work tackles finite-time suboptimality in online control of discrete-time nonlinear time-varying systems by bounding the extra cost incurred by suboptimal policies relative to a benchmark policy. It introduces and leverages exponential incremental stability ($*ediss$) to derive tight finite-time suboptimality bounds that depend on the suboptimal trajectory pathlength and the benchmark’s convergence rate. The authors prove that exponential stability of the benchmark (even in non-smooth settings) implies $*ediss$ under mild local conditions, enabling accurate transient guarantees for online controllers like suboptimal MPC. A detailed model-predictive-control use case with a suboptimal projected gradient method demonstrates the bounds and highlights a trade-off between computation and suboptimality. The framework provides actionable insights for allocating computational resources in real-time control and has potential applications in adaptive and online control where suboptimality arises from parameter uncertainty or changing costs.

Abstract

Suboptimal methods in optimal control arise due to a limited computational budget, unknown system dynamics, or a short prediction window among other reasons. Although these methods are ubiquitous, their transient performance remains relatively unstudied. We consider the control of discrete-time, nonlinear time-varying dynamical systems and establish sufficient conditions to analyze the finite-time closed-loop performance of such methods in terms of the additional cost incurred due to suboptimality. Finite-time guarantees allow the control design to distribute a limited computational budget over a time horizon and estimate the on-the-go loss in performance due to suboptimality. We study exponential incremental input-to-state stabilizing policies and show that for nonlinear systems, under some mild conditions, this property is directly implied by exponential stability without further assumptions on global smoothness. The analysis is showcased on a suboptimal model predictive control use case.

Closed-Loop Finite-Time Analysis of Suboptimal Online Control

TL;DR

This work tackles finite-time suboptimality in online control of discrete-time nonlinear time-varying systems by bounding the extra cost incurred by suboptimal policies relative to a benchmark policy. It introduces and leverages exponential incremental stability () to derive tight finite-time suboptimality bounds that depend on the suboptimal trajectory pathlength and the benchmark’s convergence rate. The authors prove that exponential stability of the benchmark (even in non-smooth settings) implies under mild local conditions, enabling accurate transient guarantees for online controllers like suboptimal MPC. A detailed model-predictive-control use case with a suboptimal projected gradient method demonstrates the bounds and highlights a trade-off between computation and suboptimality. The framework provides actionable insights for allocating computational resources in real-time control and has potential applications in adaptive and online control where suboptimality arises from parameter uncertainty or changing costs.

Abstract

Suboptimal methods in optimal control arise due to a limited computational budget, unknown system dynamics, or a short prediction window among other reasons. Although these methods are ubiquitous, their transient performance remains relatively unstudied. We consider the control of discrete-time, nonlinear time-varying dynamical systems and establish sufficient conditions to analyze the finite-time closed-loop performance of such methods in terms of the additional cost incurred due to suboptimality. Finite-time guarantees allow the control design to distribute a limited computational budget over a time horizon and estimate the on-the-go loss in performance due to suboptimality. We study exponential incremental input-to-state stabilizing policies and show that for nonlinear systems, under some mild conditions, this property is directly implied by exponential stability without further assumptions on global smoothness. The analysis is showcased on a suboptimal model predictive control use case.
Paper Structure (15 sections, 19 theorems, 106 equations, 3 figures)

This paper contains 15 sections, 19 theorems, 106 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries and Problem Setup
  3. Suboptimality Gap Analysis
  4. Upper Bound
  5. Interpretation of the Upper Bound
  6. Exponentially Stable Policies and *ediss
  7. Preliminaries on Exponential Stability
  8. Preliminaries on Exponential Incremental Stability
  9. Main Results
  10. Model Predictive Control - A Use Case
  11. Optimal MPC
  12. Suboptimal MPC
  13. Suboptimality Gap
  14. Numerical Example
  15. Conclusions

Key Result

Theorem 1

Let Assumptions assum:mu_star, assum:Lipschitz_g and assum:stage_cost hold, then the suboptimality gap of any policy, $\mu$, fulfilling Assumption assum:mu_contractive satisfies for all $x_0 \in \mathcal{D}^\mu$. Specifically, it is bounded by for all $x_0 \in \mathcal{D}^\mu$, where $\delta u_0 := \nu - \mu_0^\star(x_0)$ and $\bar{M} := \left(M_u+\frac{c_wL_u\left(M_uL+M_x\right)}{1-\rho}\right

Figures (3)

  • Figure 1: Two separate closed-loop trajectories, generated by applying a suboptimal input signal $\boldsymbol{u}^\mu$, and a benchmark input signal $\boldsymbol{u^\star}$.
  • Figure 2: The pictorial evolution of suboptimal and benchmark trajectories evolving in $\mathcal{D}^\mu \subseteq \mathcal{D}^\star$.
  • Figure 3: The phase plot on the left shows two separate trajectories generated by applying respectively TD-MPC, with a constant $\ell=6$ (in green) and optimal MPC policies (in purple). The logarithmic scale plot on the right shows the empirical suboptimality gap (in green), as well as the order of the upper bound decrease (in red) as $\ell$, and therefore the computation time is increased. The two curves on the right are parameterized by $\ell$, ranging from $1$ to $5000$.

Theorems & Definitions (37)

  • Definition 1
  • Theorem 1
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • proof : Proof of Theorem \ref{['the:incurred_suboptimality']}
  • Corollary 1
  • ...and 27 more