Uncoupled and Convergent Learning in Monotone Games under Bandit Feedback

TL;DR

The first non-asymptotic result in converging monotone games is provided and the algorithm is extended to time-varying monotone games and gives improved results for equilibrium tracking games.

Abstract

We study the problem of no-regret learning algorithms for general monotone and smooth games and their last-iterate convergence properties. Specifically, we investigate the problem under bandit feedback and strongly uncoupled dynamics, which allows modular development of the multi-player system that applies to a wide range of real applications. We propose a mirror-descent-based algorithm, which converges in $O(T^{-1/4})$ and is also no-regret. The result is achieved by a dedicated use of two regularizations and the analysis of the fixed point thereof. The convergence rate is further improved to $O(T^{-1/2})$ in the case of strongly monotone games. Motivated by practical tasks where the game evolves over time, the algorithm is extended to time-varying monotone games. We provide the first non-asymptotic result in converging monotone games and give improved results for equilibrium tracking games.

Figures (3)

• Figure 1: Experiment on Cournot competition, zero-sum two-player minimax game, and monotone-concave game.
• Figure 2: More examples on the zero-sum matrix game, with $A$ being $[2, 1], [1, 3]$, $[3, 0], [0, 1]$, and $[1, 2], [2, 0]$.
• Figure 3: More examples on the Cournot competition, with the marginal cost being $50, 60, 70$.

Theorems & Definitions (42)

• Definition 3.1: Nash equilibrium
• Example 3.1: monotone-concave game
• Example 3.2: Cournot competition
• Example 3.3: Splittable routing game
• Definition 4.1
• Theorem 5.1
• Theorem 5.2
• Theorem 5.3
• Definition 5.1: roughgarden2015intrinsicsyrgkanis2015fast
• Proposition 5.1