Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Safe Control and Learning Using the Generalized Action Governor

Nan Li, Yutong Li, Ilya Kolmanovsky, Anouck Girard, H. Eric Tseng, Dimitar Filev

TL;DR

A general framework for safe control and learning based on the generalized action governor (AG), which aims at safely evolving control parameters using real-time data to enhance control performance in uncertain systems is introduced.

Abstract

This article introduces a general framework for safe control and learning based on the generalized action governor (AG). The AG is a supervisory scheme for augmenting a nominal closed-loop system with the ability of strictly handling prescribed safety constraints. In the first part of this article, we present a generalized AG methodology and analyze its key properties in a general setting. Then, we introduce tailored AG design approaches derived from the generalized methodology for linear and discrete systems. Afterward, we discuss the application of the generalized AG to facilitate safe online learning, which aims at safely evolving control parameters using real-time data to enhance control performance in uncertain systems. We present two safe learning algorithms based on, respectively, reinforcement learning and data-driven Koopman operator-based control integrated with the generalized AG to exemplify this application. Finally, we illustrate the developments with a numerical example.

Safe Control and Learning Using the Generalized Action Governor

TL;DR

A general framework for safe control and learning based on the generalized action governor (AG), which aims at safely evolving control parameters using real-time data to enhance control performance in uncertain systems is introduced.

Abstract

This article introduces a general framework for safe control and learning based on the generalized action governor (AG). The AG is a supervisory scheme for augmenting a nominal closed-loop system with the ability of strictly handling prescribed safety constraints. In the first part of this article, we present a generalized AG methodology and analyze its key properties in a general setting. Then, we introduce tailored AG design approaches derived from the generalized methodology for linear and discrete systems. Afterward, we discuss the application of the generalized AG to facilitate safe online learning, which aims at safely evolving control parameters using real-time data to enhance control performance in uncertain systems. We present two safe learning algorithms based on, respectively, reinforcement learning and data-driven Koopman operator-based control integrated with the generalized AG to exemplify this application. Finally, we illustrate the developments with a numerical example.
Paper Structure (12 sections, 3 theorems, 33 equations, 4 figures, 3 algorithms)

This paper contains 12 sections, 3 theorems, 33 equations, 4 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Models and Assumptions
  3. Generalized Action Governor
  4. Generalized action governor for linear systems
  5. Generalized action governor for discrete systems
  6. Safe Online Learning
  7. Safe $Q$-learning
  8. Safe data-driven Koopman control
  9. Illustrative Example
  10. Action governor design and safe control
  11. Safe online learning using action governor
  12. Conclusions

Key Result

Proposition 1

If equ:AG_2 is feasible at the initial time $t = 0$, then the system trajectory $(x(t), u(t))$ under the AG supervision satisfies the constraints in equ:constraint for all $t \in \mathbb{Z}_+$.

Figures (4)

  • Figure 1: Safe online learning architecture.
  • Figure 2: State trajectories under nominal control without AG, nominal control with AG, nominal control with CBF, and learned Koopman control with AG.
  • Figure 3: Safe sets computed using linear systems approach versus discrete systems approach.
  • Figure 4: Average cost during learning.

Theorems & Definitions (5)

  • Proposition 1: All-Time Safety
  • Proposition 2: Eventual Feasibility
  • Proposition 3: Recursive Feasibility
  • Remark 1: Use Case
  • Remark 2: Use Case