Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning generalized Nash equilibria from pairwise preferences

Pablo Krupa, Alberto Bemporad

Abstract

Generalized Nash Equilibrium Problems (GNEPs) arise in many applications, including non-cooperative multi-agent control problems. Although many methods exist for finding generalized Nash equilibria, most of them rely on assuming knowledge of the objective functions or being able to query the best responses of the agents. We present a method for learning solutions of GNEPs only based on querying agents for their preference between two alternative decisions. We use the collected preference data to learn a GNEP whose equilibrium approximates a GNE of the underlying (unknown) problem. Preference queries are selected using an active-learning strategy that balances exploration of the decision space and exploitation of the learned GNEP. We present numerical results on game-theoretic linear quadratic regulation problems, as well as on other literature GNEP examples, showing the effectiveness of the proposed method.

Learning generalized Nash equilibria from pairwise preferences

Abstract

Generalized Nash Equilibrium Problems (GNEPs) arise in many applications, including non-cooperative multi-agent control problems. Although many methods exist for finding generalized Nash equilibria, most of them rely on assuming knowledge of the objective functions or being able to query the best responses of the agents. We present a method for learning solutions of GNEPs only based on querying agents for their preference between two alternative decisions. We use the collected preference data to learn a GNEP whose equilibrium approximates a GNE of the underlying (unknown) problem. Preference queries are selected using an active-learning strategy that balances exploration of the decision space and exploitation of the learned GNEP. We present numerical results on game-theoretic linear quadratic regulation problems, as well as on other literature GNEP examples, showing the effectiveness of the proposed method.
Paper Structure (7 sections, 20 equations, 2 figures, 2 tables)

This paper contains 7 sections, 20 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Learning a GNE from preferences
  3. Active learning method for finding a GNE
  4. Numerical results
  5. Game-theoretic linear quadratic regulator
  6. Solving GNEPs taken from the literature
  7. Conclusions

Figures (2)

  • Figure 1: Results of Algorithm \ref{['alg:AL']} for game-theoretic LQR problem with $n_\xi = m = 12$ and $N = 4$.
  • Figure 2: Solving GNEPs taken from the literature. Figures show the values of $x^k$ obtained from solving \ref{['eq:GNEPi']} for the $\theta_i^k$ learned at each iteration $k$ of Algorithm \ref{['alg:AL']} (solid lines) converging towards a GNE of the underlying GNEP (dashed lines).

Theorems & Definitions (4)

  • Remark 1: On the existence of a GNE for problem \ref{['eq:GNEPi']}
  • Remark 2: On the inclusion of the dissimilarity functions
  • Remark 3: On adding noise to $x_i^{k, 2}$
  • Remark 4: Minimum values of $\delta^k$ and $\sigma^k$