Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Nonstochastic Control with Convex Safety Constraints

Nanfei Jiang, Spencer Hutchinson, Mahnoosh Alizadeh

TL;DR

This work addresses online nonstochastic control for linear time-invariant systems under general convex safety constraints and adversarial time-varying costs. It introduces the Online Gradient Descent with Buffer Zone for Convex Constraints (OGD-BZC), which couples a disturbance-action controller with projected online gradient descent over DAC weights to guarantee safety and achieve sublinear regret. Theoretical results establish deterministic safety under carefully chosen horizon, buffer, and stepsize parameters, along with a $\tilde{O}(\sqrt{T})$ regret bound that is, notably, independent of the number of individual constraints. Numerical experiments on a two-dimensional toy example illustrate robust performance and safety under adversarial disturbances, underscoring the approach’s practicality for safe online control in convex constraint settings.

Abstract

This paper considers the online nonstochastic control problem of a linear time-invariant system under convex state and input constraints that need to be satisfied at all times. We propose an algorithm called Online Gradient Descent with Buffer Zone for Convex Constraints (OGD-BZC), designed to handle scenarios where the system operates within general convex safety constraints. We demonstrate that OGD-BZC, with appropriate parameter selection, satisfies all the safety constraints under bounded adversarial disturbances. Additionally, to evaluate the performance of OGD-BZC, we define the regret with respect to the best safe linear policy in hindsight. We prove that OGD-BZC achieves $\tilde{O} (\sqrt{T})$ regret given proper parameter choices. Our numerical results highlight the efficacy and robustness of the proposed algorithm.

Online Nonstochastic Control with Convex Safety Constraints

TL;DR

This work addresses online nonstochastic control for linear time-invariant systems under general convex safety constraints and adversarial time-varying costs. It introduces the Online Gradient Descent with Buffer Zone for Convex Constraints (OGD-BZC), which couples a disturbance-action controller with projected online gradient descent over DAC weights to guarantee safety and achieve sublinear regret. Theoretical results establish deterministic safety under carefully chosen horizon, buffer, and stepsize parameters, along with a regret bound that is, notably, independent of the number of individual constraints. Numerical experiments on a two-dimensional toy example illustrate robust performance and safety under adversarial disturbances, underscoring the approach’s practicality for safe online control in convex constraint settings.

Abstract

This paper considers the online nonstochastic control problem of a linear time-invariant system under convex state and input constraints that need to be satisfied at all times. We propose an algorithm called Online Gradient Descent with Buffer Zone for Convex Constraints (OGD-BZC), designed to handle scenarios where the system operates within general convex safety constraints. We demonstrate that OGD-BZC, with appropriate parameter selection, satisfies all the safety constraints under bounded adversarial disturbances. Additionally, to evaluate the performance of OGD-BZC, we define the regret with respect to the best safe linear policy in hindsight. We prove that OGD-BZC achieves regret given proper parameter choices. Our numerical results highlight the efficacy and robustness of the proposed algorithm.

Paper Structure

This paper contains 16 sections, 17 theorems, 112 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Basic Notations
  3. Problem Formulation
  4. Preliminaries
  5. The OGD-BZC Algorithm
  6. Theoretical Result
  7. Safety Result
  8. Regret Analysis
  9. Proof of Theorem \ref{['thm: regret bound']}
  10. Numerical Experiments
  11. Conclusion
  12. Acknowledgements
  13. Supporting Lemmas
  14. Proofs related to Theorem \ref{['thm: safety']}
  15. Proof related to Theorem \ref{['thm: regret bound']}
  16. ...and 1 more sections

Key Result

Proposition 1

When applying a DAC $\pi(K,\{\bm M_t\})$ as shown in def: DAC, the state and control input can be written as: where $A_K = A - BK$ and the superscript $H$ denotes raising the matrix to the $H$-th power, $\tilde{x}_t$ and $\tilde{u}_t$ are referred to as the surrogate state and input and are defined as: where $\bm M_{t-H:t} := \left \{ \bm M_{t-H},...,\bm M_t \right \}$ and the "disturbance-to-

Figures (2)

  • Figure 1: State Evolution of OGD-BZC and Stable linear controller with horizon $T = 30$.
  • Figure 2: Average regret of OGD-BZC.

Theorems & Definitions (35)

  • Definition 1: Strong Stability
  • Definition 2: Shrinkage and Expansion hutchinson_impact_2023
  • Definition 3: Strictly and Loosely Safe Policies
  • Definition 4: Disturbance-Action Controller (DAC)
  • Proposition 1: Lemma 4.3 in agarwal_online_2019
  • Theorem 1
  • Theorem 2: Regret Bound
  • Corollary 1: Sublinear Regret
  • Lemma 1
  • Lemma 2
  • ...and 25 more