Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pointwise Convergence in Games with Conflicting Interest

Nanxiang Zhou, Jing Dong, Baoxiang Wang

TL;DR

The paper develops non-negative weighted regret to unify analyses of harmonic and zero-sum games, enabling a common framework for learning dynamics in conflicting-interest settings. It analyzes optimistic no-regret algorithms, specifically OMD and OFTRL, proving they reach an $ε$-approximate Nash equilibrium in $O(1/ε^2)$ iterations and exhibit pointwise convergence to NE when the NE set is finite. The results extend to corrupted dynamics with finite deviations, retaining convergence guarantees and showing equivalence between corrupted OMD and OFTRL. Empirical evaluations on Matching Pennies and harmonic games corroborate the theory, illustrating practical convergence behavior and robustness of the proposed methods.

Abstract

In this work, we introduce the concept of non-negative weighted regret, an extension of non-negative regret \cite{anagnostides2022last} in games. Investigating games with non-negative weighted regret helps us to understand games with conflicting interests, including harmonic games and important classes of zero-sum games.We show that optimistic variants of classical no-regret learning algorithms, namely optimistic mirror descent (OMD) and optimistic follow the regularized leader (OFTRL), converge to an $ε$-approximate Nash equilibrium at a rate of $O(1/ε^2)$.Consequently, they guarantee pointwise convergence to a Nash equilibrium if there are only finitely many Nash equilibria in the game. These algorithms are robust in the sense the convergence holds even if the players deviate Our theoretical findings are supported by empirical evaluations of OMD and OFTRL on the game of matching pennies and harmonic game instances.

Pointwise Convergence in Games with Conflicting Interest

TL;DR

The paper develops non-negative weighted regret to unify analyses of harmonic and zero-sum games, enabling a common framework for learning dynamics in conflicting-interest settings. It analyzes optimistic no-regret algorithms, specifically OMD and OFTRL, proving they reach an -approximate Nash equilibrium in iterations and exhibit pointwise convergence to NE when the NE set is finite. The results extend to corrupted dynamics with finite deviations, retaining convergence guarantees and showing equivalence between corrupted OMD and OFTRL. Empirical evaluations on Matching Pennies and harmonic games corroborate the theory, illustrating practical convergence behavior and robustness of the proposed methods.

Abstract

In this work, we introduce the concept of non-negative weighted regret, an extension of non-negative regret \cite{anagnostides2022last} in games. Investigating games with non-negative weighted regret helps us to understand games with conflicting interests, including harmonic games and important classes of zero-sum games.We show that optimistic variants of classical no-regret learning algorithms, namely optimistic mirror descent (OMD) and optimistic follow the regularized leader (OFTRL), converge to an -approximate Nash equilibrium at a rate of .Consequently, they guarantee pointwise convergence to a Nash equilibrium if there are only finitely many Nash equilibria in the game. These algorithms are robust in the sense the convergence holds even if the players deviate Our theoretical findings are supported by empirical evaluations of OMD and OFTRL on the game of matching pennies and harmonic game instances.

Paper Structure

This paper contains 21 sections, 31 theorems, 80 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Learning in games with non-negative regrets and zero-sum games
  4. Learning in harmonic games
  5. Preliminaries
  6. Notation
  7. Normal-Form Game
  8. Solution Concepts
  9. Harmonic Games
  10. No-regret Learning
  11. OMD
  12. OFTRL
  13. Games with Non-negative Weighted Regrets
  14. Convergence in Games with Non-negative Weighted Regret
  15. Learning with Corruptions
  16. ...and 6 more sections

Key Result

Lemma 5.1

Harmonic games have non-negative weighted regrets.

Figures (2)

  • Figure 1: Total regret plot in a Harmonic game where the utility is the collective sum of action minus the individual action. Each point represents the total regret after $10$ rounds.
  • Figure 2: The two plots illustrate the convergence of OMD and OFTRL algorithms towards the Nash equilibrium in the Matching Pennies and the Harmonic game over $100$ iterations.

Theorems & Definitions (58)

  • Definition 3.1: Nash equilibrium
  • Definition 3.2: $\epsilon$-approximate Nash equilibrium
  • Definition 3.3: abdou2022decompositionlegacci2024no
  • Definition 4.1
  • Definition 5.1
  • Definition 5.2
  • Lemma 5.1
  • Lemma 5.2: Extension of Proposition 3.2 of anagnostides2022last
  • proof
  • Theorem 6.1
  • ...and 48 more