Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On Persistently Resetting Learning Integrators: A Framework For Model-Free Feedback Optimization

Mahmoud Abdelgalil, Jorge I. Poveda

TL;DR

The paper introduces persistently resetting learning integrators (PRLI) as a novel model-free, zeroth-order gradient oracle for real-time feedback optimization in dynamical systems. By formulating the optimization-and-plant interconnection as a hybrid dynamical system (HDS) and leveraging unbiased persistently exciting signals, PRLIs deliver global, real-time gradient estimates with controllably small perturbations, enabling robust coupling with discrete-time optimization algorithms. The authors establish uniform global asymptotic stability (UGAS) for static cost functions and extend the results to dynamic costs via a singular-perturbation timescale separation, proving semi-global practical stability (SGPAS) as the plant fast dynamics decouple. Numerical simulations validate the framework, demonstrating convergence of gradient descent, heavy-ball, and projected gradient descent under various PRLI-based configurations and showing effective interconnection with dynamic plants. Overall, the work bridges continuous-time physical dynamics with discrete-time optimization, offering a modular, globally stabilizing approach to model-free feedback optimization.

Abstract

We study a novel class of algorithms for solving model-free feedback optimization problems in dynamical systems. The key novelty is the introduction of \emph{persistent resetting learning integrators} (PRLI), which are integrators that are reset at the same frequency at which the plant is dithered using exploratory signals for model-free optimization. It is shown that PRLIs can serve as core mechanisms for real-time gradient estimation in online feedback-optimization tasks where only cost function measurements are available. In particular, unlike existing approaches based on approximation theory, such as averaging or finite-differences, PRLIs can produce global real-time gradient estimates of cost functions, with uniformly bounded perturbations of arbitrarily small magnitude. In this sense, PRLIs function as robust \emph{hybrid} "Oracles" suitable for interconnection with discrete-time optimization algorithms that optimize the performance of continuous-time dynamical plants in closed-loop operation. Compared to existing methods, PRLIs yield \emph{global} stability properties for a broad class of cost functions, surpassing the local or semi-global guarantees offered by traditional approaches based on perturbation and approximation theory. The proposed framework naturally bridges physical systems, modeled as continuous-time plants where continuous exploration is essential, with digital algorithms, represented as discrete-time optimization methods. The main results are illustrated using different numerical examples.

On Persistently Resetting Learning Integrators: A Framework For Model-Free Feedback Optimization

TL;DR

The paper introduces persistently resetting learning integrators (PRLI) as a novel model-free, zeroth-order gradient oracle for real-time feedback optimization in dynamical systems. By formulating the optimization-and-plant interconnection as a hybrid dynamical system (HDS) and leveraging unbiased persistently exciting signals, PRLIs deliver global, real-time gradient estimates with controllably small perturbations, enabling robust coupling with discrete-time optimization algorithms. The authors establish uniform global asymptotic stability (UGAS) for static cost functions and extend the results to dynamic costs via a singular-perturbation timescale separation, proving semi-global practical stability (SGPAS) as the plant fast dynamics decouple. Numerical simulations validate the framework, demonstrating convergence of gradient descent, heavy-ball, and projected gradient descent under various PRLI-based configurations and showing effective interconnection with dynamic plants. Overall, the work bridges continuous-time physical dynamics with discrete-time optimization, offering a modular, globally stabilizing approach to model-free feedback optimization.

Abstract

We study a novel class of algorithms for solving model-free feedback optimization problems in dynamical systems. The key novelty is the introduction of \emph{persistent resetting learning integrators} (PRLI), which are integrators that are reset at the same frequency at which the plant is dithered using exploratory signals for model-free optimization. It is shown that PRLIs can serve as core mechanisms for real-time gradient estimation in online feedback-optimization tasks where only cost function measurements are available. In particular, unlike existing approaches based on approximation theory, such as averaging or finite-differences, PRLIs can produce global real-time gradient estimates of cost functions, with uniformly bounded perturbations of arbitrarily small magnitude. In this sense, PRLIs function as robust \emph{hybrid} "Oracles" suitable for interconnection with discrete-time optimization algorithms that optimize the performance of continuous-time dynamical plants in closed-loop operation. Compared to existing methods, PRLIs yield \emph{global} stability properties for a broad class of cost functions, surpassing the local or semi-global guarantees offered by traditional approaches based on perturbation and approximation theory. The proposed framework naturally bridges physical systems, modeled as continuous-time plants where continuous exploration is essential, with digital algorithms, represented as discrete-time optimization methods. The main results are illustrated using different numerical examples.

Paper Structure

This paper contains 16 sections, 120 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Notation, Definitions, and Preliminaries
  3. Notation
  4. Hybrid Dynamical Systems
  5. Boundedness and Stability Notions for HDS
  6. Unbiased Persistently Exciting Functions
  7. Algorithm Description: Static Maps
  8. PRLIs as Gradient Oracles: The Main Idea
  9. Formulation as Hybrid Systems
  10. Input-to-State Stability of Discrete-Time Optimization Schemes
  11. Input-to-State Stability of Gradient Descent
  12. Input-to-State Stability of Heavy Ball Method
  13. Input-to-State Stability of Projected Gradient Descent
  14. Interconnections with Dynamic Plants
  15. Numerical Simulations
  16. ...and 1 more sections

Figures (7)

  • Figure 1: (a): A block diagram description of the interconnection between the proposed model-free optimization algorithm and a static cost function. (b): a 3D sketch of the behavior of the optimization parameter $u$ and the cost $\phi(u)$ under the HDS $\mathcal{H}$. (c): a 2D sketch of the typical behavior of the optimization parameter $u$ under the HDS $\mathcal{H}$.
  • Figure 2: A block diagram of the discrete-time system \ref{['eq:discrete-time-system-with-input']}.
  • Figure 3: A block diagram description of the interconnection between the proposed model-free optimization algorithm and a dynamic cost function arising as the steady-state output function of a "fast" plant.
  • Figure 4: The UPE function defined in \ref{['eq:exploration_signal_example_1']}.
  • Figure 5: Simulation results for Example 1. The top row shows the evolution of components of the parameter $u$. The bottom row shows the evolution of the components of the state $p$ of the PRLI. The insets in the bottom row show the state $p$ of the PRLI immediately before a jump is triggered, which resets $p$ to $p=0$.
  • ...and 2 more figures

Theorems & Definitions (8)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Example 1
  • Example 2
  • Example 3