Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The computation of approximate feedback Stackelberg equilibria in multi-player nonlinear constrained dynamic games

Jingqi Li, Somayeh Sojoudi, Claire Tomlin, David Fridovich-Keil

TL;DR

This work tackles the challenge of computing approximate local feedback Stackelberg equilibria in multi-player constrained dynamic games with nonlinear dynamics. It introduces a nested-optimization reformulation that yields KKT conditions and a second-order sufficient condition, and develops a Newton-style primal-dual interior-point method for constrained $LQ$ games, extending it to nonlinear games via iterative LQ approximations with quasi-policies and proving exponential convergence. The authors provide both theoretical convergence guarantees and empirical validation in a highway lane-merging scenario, showing robustness to infeasible starts and rapid convergence. Overall, the paper delivers a scalable, provably convergent framework for constrained FSE computation relevant to autonomous driving, multi-agent control, and related domains.

Abstract

Solving feedback Stackelberg games with nonlinear dynamics and coupled constraints, a common scenario in practice, presents significant challenges. This work introduces an efficient method for computing approximate local feedback Stackelberg equilibria in multi-player general-sum dynamic games, with continuous state and action spaces. Different from existing (approximate) dynamic programming solutions that are primarily designed for unconstrained problems, our approach involves reformulating a feedback Stackelberg dynamic game into a sequence of nested optimization problems, enabling the derivation of Karush-Kuhn-Tucker (KKT) conditions and the establishment of a second-order sufficient condition for local feedback Stackelberg equilibria. We propose a Newton-style primal-dual interior point method for solving constrained linear quadratic (LQ) feedback Stackelberg games, offering provable convergence guarantees. Our method is further extended to compute local feedback Stackelberg equilibria for more general nonlinear games by iteratively approximating them using LQ games, ensuring that their KKT conditions are locally aligned with those of the original nonlinear games. We prove the exponential convergence of our algorithm in constrained nonlinear games. In a feedback Stackelberg game with nonlinear dynamics and (nonconvex) coupled costs and constraints, our experimental results reveal the algorithm's ability to handle infeasible initial conditions and achieve exponential convergence towards an approximate local feedback Stackelberg equilibrium.

The computation of approximate feedback Stackelberg equilibria in multi-player nonlinear constrained dynamic games

TL;DR

This work tackles the challenge of computing approximate local feedback Stackelberg equilibria in multi-player constrained dynamic games with nonlinear dynamics. It introduces a nested-optimization reformulation that yields KKT conditions and a second-order sufficient condition, and develops a Newton-style primal-dual interior-point method for constrained games, extending it to nonlinear games via iterative LQ approximations with quasi-policies and proving exponential convergence. The authors provide both theoretical convergence guarantees and empirical validation in a highway lane-merging scenario, showing robustness to infeasible starts and rapid convergence. Overall, the paper delivers a scalable, provably convergent framework for constrained FSE computation relevant to autonomous driving, multi-agent control, and related domains.

Abstract

Solving feedback Stackelberg games with nonlinear dynamics and coupled constraints, a common scenario in practice, presents significant challenges. This work introduces an efficient method for computing approximate local feedback Stackelberg equilibria in multi-player general-sum dynamic games, with continuous state and action spaces. Different from existing (approximate) dynamic programming solutions that are primarily designed for unconstrained problems, our approach involves reformulating a feedback Stackelberg dynamic game into a sequence of nested optimization problems, enabling the derivation of Karush-Kuhn-Tucker (KKT) conditions and the establishment of a second-order sufficient condition for local feedback Stackelberg equilibria. We propose a Newton-style primal-dual interior point method for solving constrained linear quadratic (LQ) feedback Stackelberg games, offering provable convergence guarantees. Our method is further extended to compute local feedback Stackelberg equilibria for more general nonlinear games by iteratively approximating them using LQ games, ensuring that their KKT conditions are locally aligned with those of the original nonlinear games. We prove the exponential convergence of our algorithm in constrained nonlinear games. In a feedback Stackelberg game with nonlinear dynamics and (nonconvex) coupled costs and constraints, our experimental results reveal the algorithm's ability to handle infeasible initial conditions and achieve exponential convergence towards an approximate local feedback Stackelberg equilibrium.
Paper Structure (22 sections, 8 theorems, 66 equations, 8 figures, 1 algorithm)

This paper contains 22 sections, 8 theorems, 66 equations, 8 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Constrained Feedback Stackelberg Games
  4. Local Feedback Stackelberg Equilibria
  5. Necessary and Sufficient Conditions for Local Feedback Stackelberg Equilibria
  6. Constrained Linear Quadratic Games
  7. Computing Feedback Stackelberg Equilibria and Constructing the KKT Conditions for LQ Games
  8. Player N at the T-th stage
  9. Players i<N at the T-th stage
  10. Player N at a stage t<T
  11. Players i<N at a stage t<T
  12. Primal-Dual Interior Point Algorithm and Convergence Analysis in Constrained LQ Games
  13. From LQ Games to Nonlinear Games
  14. Iteratively Approximating Nonlinear Games via LQ Games by Aligning Their KKT Conditions
  15. Quasi-Policies Approximation Error and Exponential Convergence Analysis in Nonlinear Games
  16. ...and 7 more sections

Key Result

Theorem 4.1

\newlabelthm:nested constrained optimization problem0 Under Assumption assumption:feasible, for each $t\in \mathbf{I}_0^T$ and each $i\in\mathbf{I}_1^N$, a local feedback Stackelberg policy $\pi_t^{i}$ can be equivalently represented as an optimization problem, given the knowledge of current state where we drop eq:subsequent player's strategy when $i=N$, and we drop eq:future player's strategy, e

Figures (8)

  • Figure 1: $\rho = 1$.
  • Figure 1: Trajectories
  • Figure 2: $\rho = 2^{-1}$.
  • Figure 3: $\rho = 2^{-5}$.
  • Figure 4: $\rho = 2^{-10}$.
  • ...and 3 more figures

Theorems & Definitions (25)

  • Definition 3.2: Local Feedback Stackelberg Equilibria basar1999dynamic
  • Remark 3.3: Existence of Local Feedback Stackelberg Equilibria
  • Theorem 4.1
  • Proof 1
  • Theorem 4.2: Necessary Condition
  • Proof 2
  • Theorem 4.3: Sufficient Condition
  • Proof 3
  • Remark 4.4
  • Theorem 5.1
  • ...and 15 more