Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nested Extremum Seeking Converges to Stackelberg Equilibrium

Brad Ratto, Alan Williams, Miroslav Krstić, Tamer Başar, Alexander Scheinker

Abstract

The nested Extremum Seeking (nES) algorithm is a model-free optimization method that has been shown to converge to a neighborhood of a Nash equilibrium. In this work, we demonstrate that the same nES dynamics can instead be made to converge to a neighborhood of a Stackelberg (leader--follower) equilibrium by imposing a different scaling law on the algorithm's design parameters. For the two--level nested case, using Lie--bracket averaging and singular perturbation arguments, we provide a rigorous stability proof showing semi-global practical asymptotic convergence to a Stackelberg equilibrium under appropriate time-scale separation. The results reveal that equilibrium selection, Nash versus Stackelberg, depends not on modifying the closed-loop dynamics, but on the hierarchical scaling of design parameters and the induced time-scale structure. We demonstrate this effect using a simple quadratic example and the canonical Fish War game. The Stackelberg variant of nES provides a model-free framework for hierarchical optimization in multi-time-scale systems, with potential applications in power grids, networked dynamical systems, and tuning of particle accelerators.

Nested Extremum Seeking Converges to Stackelberg Equilibrium

Abstract

The nested Extremum Seeking (nES) algorithm is a model-free optimization method that has been shown to converge to a neighborhood of a Nash equilibrium. In this work, we demonstrate that the same nES dynamics can instead be made to converge to a neighborhood of a Stackelberg (leader--follower) equilibrium by imposing a different scaling law on the algorithm's design parameters. For the two--level nested case, using Lie--bracket averaging and singular perturbation arguments, we provide a rigorous stability proof showing semi-global practical asymptotic convergence to a Stackelberg equilibrium under appropriate time-scale separation. The results reveal that equilibrium selection, Nash versus Stackelberg, depends not on modifying the closed-loop dynamics, but on the hierarchical scaling of design parameters and the induced time-scale structure. We demonstrate this effect using a simple quadratic example and the canonical Fish War game. The Stackelberg variant of nES provides a model-free framework for hierarchical optimization in multi-time-scale systems, with potential applications in power grids, networked dynamical systems, and tuning of particle accelerators.

Paper Structure

This paper contains 19 sections, 7 theorems, 56 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Strong Monotonicity and Stability Lemma
  4. CTP, GUAS, and SPUAS Definitions
  5. Lie--Bracket Averaging
  6. GUAS $+$ CTP $\Rightarrow$ SPUAS
  7. nES Dynamics and Assumptions
  8. System Approximations
  9. 2D Lie-Bracket Averaging
  10. Singular Perturbation
  11. Standard Singular Perturbation Form
  12. State Transformation
  13. Boundary-Layer Model
  14. The Reduced Order Model and Quasi--Steady State
  15. 1D Lie--Bracket Averaging
  16. ...and 4 more sections

Key Result

Lemma 1

Consider $q(x):\mathbb{R}\to\mathbb R$ to be continuously differentiable and $\mu$--strongly monotone on $\mathbb{R}$ for some $\mu>0$. Let $q(x^*)=0$ for some $x^*\in\mathbb{R}$. Then, $x^*$ is the unique exponentially stable equilibrium of the system where $c>0$. In particular,

Figures (7)

  • Figure 1: Block diagram of the nES architecture for $n=2$ nestings. The Slow ES Leader tunes $x_1$, and the Fast ES Follower tunes $x_2$.
  • Figure 2: The system approximations used in the analysis, from the original nES dynamics (top) to the averaged ROM (bottom).
  • Figure 3: Hierarchical nES time-scale diagram for the $n=2$ nested case.
  • Figure 4: Phase space plot for the example discussed in Section \ref{['sec:Simple-Example']}, comparing the practical convergence to the Nash (brown traces) versus Stackelberg (grey traces) equilibrium under the nES dynamics.
  • Figure 5: Phase space plot for the Fish War example discussed in Section \ref{['sec:fishwar']}, comparing the practical convergence to Nash (brown traces) versus Stackelberg (grey traces) equilibrium under the nES dynamics.
  • ...and 2 more figures

Theorems & Definitions (13)

  • Definition 1: Strong monotonicity
  • Lemma 1
  • proof
  • Definition 2: CTP
  • Definition 3: GUAS
  • Definition 4: SPUAS
  • Theorem 1
  • Theorem 2
  • Proposition 1
  • Proposition 2
  • ...and 3 more