Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerated regularized learning in finite N-person games

Kyriakos Lotidis, Angeliki Giannou, Panayotis Mertikopoulos, Nicholas Bambos

TL;DR

The paper investigates whether Nesterov-style acceleration can improve online learning in finite N-player games. It introduces Follow the Accelerated Leader (FTXL), a momentum-augmented regularized learning scheme that blends NAG-inspired dynamics with regularized best-response maps, and analyzes its discrete- and continuous-time behavior. The key finding is that FTXL attains local superlinear convergence to strict Nash equilibria, yielding an exponential speedup over vanilla regularized learning, and this speedup persists across full-information, realization-based, and bandit payoff-based feedback. The authors provide a concrete discrete-time algorithm, rigorous convergence guarantees under multiple information regimes, and numerical simulations in zero-sum and congestion games, highlighting the practical robustness and potential impact for fast equilibrium learning in multi-agent systems.

Abstract

Motivated by the success of Nesterov's accelerated gradient algorithm for convex minimization problems, we examine whether it is possible to achieve similar performance gains in the context of online learning in games. To that end, we introduce a family of accelerated learning methods, which we call "follow the accelerated leader" (FTXL), and which incorporates the use of momentum within the general framework of regularized learning - and, in particular, the exponential/multiplicative weights algorithm and its variants. Drawing inspiration and techniques from the continuous-time analysis of Nesterov's algorithm, we show that FTXL converges locally to strict Nash equilibria at a superlinear rate, achieving in this way an exponential speed-up over vanilla regularized learning methods (which, by comparison, converge to strict equilibria at a geometric, linear rate). Importantly, FTXL maintains its superlinear convergence rate in a broad range of feedback structures, from deterministic, full information models to stochastic, realization-based ones, and even when run with bandit, payoff-based information, where players are only able to observe their individual realized payoffs.

Accelerated regularized learning in finite N-person games

TL;DR

The paper investigates whether Nesterov-style acceleration can improve online learning in finite N-player games. It introduces Follow the Accelerated Leader (FTXL), a momentum-augmented regularized learning scheme that blends NAG-inspired dynamics with regularized best-response maps, and analyzes its discrete- and continuous-time behavior. The key finding is that FTXL attains local superlinear convergence to strict Nash equilibria, yielding an exponential speedup over vanilla regularized learning, and this speedup persists across full-information, realization-based, and bandit payoff-based feedback. The authors provide a concrete discrete-time algorithm, rigorous convergence guarantees under multiple information regimes, and numerical simulations in zero-sum and congestion games, highlighting the practical robustness and potential impact for fast equilibrium learning in multi-agent systems.

Abstract

Motivated by the success of Nesterov's accelerated gradient algorithm for convex minimization problems, we examine whether it is possible to achieve similar performance gains in the context of online learning in games. To that end, we introduce a family of accelerated learning methods, which we call "follow the accelerated leader" (FTXL), and which incorporates the use of momentum within the general framework of regularized learning - and, in particular, the exponential/multiplicative weights algorithm and its variants. Drawing inspiration and techniques from the continuous-time analysis of Nesterov's algorithm, we show that FTXL converges locally to strict Nash equilibria at a superlinear rate, achieving in this way an exponential speed-up over vanilla regularized learning methods (which, by comparison, converge to strict equilibria at a geometric, linear rate). Importantly, FTXL maintains its superlinear convergence rate in a broad range of feedback structures, from deterministic, full information models to stochastic, realization-based ones, and even when run with bandit, payoff-based information, where players are only able to observe their individual realized payoffs.
Paper Structure (27 sections, 15 theorems, 134 equations, 1 figure)

This paper contains 27 sections, 15 theorems, 134 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Finite games
  4. Regularized learning in games
  5. Combining acceleration with regularization: First insights and results
  6. NAG algorithm
  7. NAG meets FTRL
  8. First insights and results
  9. Accelerated learning: Analysis and results
  10. The algorithm
  11. Accelerated learning with full information
  12. Accelerated learning with realization-based feedback
  13. Bandit feedback
  14. Numerical Experiments
  15. Zero-sum Games.
  16. ...and 12 more sections

Key Result

Theorem 1

Let $x^{\ast}$ be a strict NE of $\Gamma$, and let $\stratof{t} = {Q}(\scoreof{t})$ be a solution orbit of eq:FTXL-cont. If $\stratof{0}$ is sufficiently close to $x^{\ast}$, then $\stratof{t}$ converges to $x^{\ast}$; in particular, if eq:FTXL-cont is run with logit best responses (that is, ${Q} \g where $C>0$ is a constant that depends only on the initialization of eq:FTXL-cont and is the minim

Figures (1)

  • Figure 1: Performance evaluation of \ref{['eq:FTXL']} in a zero-sum and a congestion game under realization-based and bandit feedback. Solid lines represent average values, while shaded regions enclose $\pm 1$ standard deviation. The plots are in logarithmic scale.

Theorems & Definitions (29)

  • Remark 1
  • Remark 2
  • Theorem 1
  • Example 3.1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Lemma A.1
  • proof
  • Lemma B.1
  • ...and 19 more