Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BIG Hype: Best Intervention in Games via Distributed Hypergradient Descent

Panagiotis D. Grontas, Giuseppe Belgioioso, Carlo Cenedese, Marta Fochesato, John Lygeros, Florian Dörfler

TL;DR

This work designs a first-order hypergradient-based algorithm for Stackelberg games and mathematically establishes its convergence using tools from nonsmooth analysis, and numerically verify the computational efficiency and scalability of the algorithm on a large-scale hierarchical demand-response model.

Abstract

Hierarchical decision making problems, such as bilevel programs and Stackelberg games, are attracting increasing interest in both the engineering and machine learning communities. Yet, existing solution methods lack either convergence guarantees or computational efficiency, due to the absence of smoothness and convexity. In this work, we bridge this gap by designing a first-order hypergradient-based algorithm for Stackelberg games and mathematically establishing its convergence using tools from nonsmooth analysis. To evaluate the \textit{hypergradient}, namely, the gradient of the upper-level objective, we develop an online scheme that simultaneously computes the lower-level equilibrium and its Jacobian. Crucially, this scheme exploits and preserves the original hierarchical and distributed structure of the problem, which renders it scalable and privacy-preserving. We numerically verify the computational efficiency and scalability of our algorithm on a large-scale hierarchical demand-response model.

BIG Hype: Best Intervention in Games via Distributed Hypergradient Descent

TL;DR

This work designs a first-order hypergradient-based algorithm for Stackelberg games and mathematically establishes its convergence using tools from nonsmooth analysis, and numerically verify the computational efficiency and scalability of the algorithm on a large-scale hierarchical demand-response model.

Abstract

Hierarchical decision making problems, such as bilevel programs and Stackelberg games, are attracting increasing interest in both the engineering and machine learning communities. Yet, existing solution methods lack either convergence guarantees or computational efficiency, due to the absence of smoothness and convexity. In this work, we bridge this gap by designing a first-order hypergradient-based algorithm for Stackelberg games and mathematically establishing its convergence using tools from nonsmooth analysis. To evaluate the \textit{hypergradient}, namely, the gradient of the upper-level objective, we develop an online scheme that simultaneously computes the lower-level equilibrium and its Jacobian. Crucially, this scheme exploits and preserves the original hierarchical and distributed structure of the problem, which renders it scalable and privacy-preserving. We numerically verify the computational efficiency and scalability of our algorithm on a large-scale hierarchical demand-response model.
Paper Structure (48 sections, 16 theorems, 76 equations, 3 figures, 2 tables, 4 algorithms)

This paper contains 48 sections, 16 theorems, 76 equations, 3 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Notation and Preliminaries
  3. Problem Formulation
  4. Lower-Level Game
  5. Upper-Level Optimization
  6. Modelling Coupling Constraints
  7. A Distributed Hypergradient Algorithm
  8. Equilibrium and Sensitivity Learning
  9. Preview of the Convergence Results
  10. Scalability in Aggregative Games
  11. Convergence Analysis
  12. Inner Loop Analysis
  13. General Games
  14. Linear-Quadratic Games (LQGs)
  15. Linear-Quadratic Subspace Games (LQSGs)
  16. ...and 33 more sections

Key Result

Proposition 1

Fix $x \in \mathcal{X}$ and assume $\mathcal{P}(x)$ is a singleton. Then, $\boldsymbol{y}^{\star}(\cdot)$ is continuously differentiable at $x$. Further, the sequence $\{\hat{s}^\ell\}_{\ell \in \mathbb{N}}$ generated by eq:s_update_nominal converges to $\boldsymbol{\mathrm{J}} \boldsymbol{y}^{\star

Figures (3)

  • Figure 1: Top: Part of a polyhedral partition $\{\tilde{\mathcal{Y}}_i \}_{i \in \mathcal{N}_p}$ of $\mathbb{R}^2$. Bottom: For LQGs, if $\boldsymbol{\omega}(x,\boldsymbol{y})$ is in the interior of $\tilde{\mathcal{Y}}_i$, then $\boldsymbol{\mathrm{J}}^c h(x, \boldsymbol{y})$ is constant and equal to the standard Jacobian $\mathbf J h(x, \boldsymbol{y})=R_i$ (coloured areas). If instead $\boldsymbol{\omega}(x,\boldsymbol{y})$ lies on the common boundary between partitions, $\boldsymbol{\mathrm{J}}^c h (x, \boldsymbol{y})$ is the convex hull of the Jacobians of the active partitions (solid white lines).
  • Figure 2: The sets $\mathcal{A}$ and $\mathcal{X}$ in \ref{['example:differentiable']}.
  • Figure 3: (a), (b) Relative suboptimality as a function of outer and total inner-loop iterations, respectively. (c) Computational time for inner and outer loop iterations versus number of followers and total problem dimension.

Theorems & Definitions (20)

  • Definition 1
  • Remark 1
  • Proposition 1
  • Lemma 1
  • Proposition 2
  • Lemma 2
  • Proposition 3
  • Lemma 3
  • Proposition 4
  • Lemma 4
  • ...and 10 more