Efficient Uncoupled Learning Dynamics with $\tilde{O}\!\left(T^{-1/4}\right)$ Last-Iterate Convergence in Bilinear Saddle-Point Problems over Convex Sets under Bandit Feedback

TL;DR

The main contribution is the design of an uncoupled learning algorithm that guarantees last-iterate convergence to the Nash equilibrium with high probability, and establishes a convergence rate of $\tilde{O}(T^{-1/4})$ up to polynomial factors in problem parameters.

Abstract

In this paper, we study last-iterate convergence of learning algorithms in bilinear saddle-point problems, a preferable notion of convergence that captures the day-to-day behavior of learning dynamics. We focus on the challenging setting where players select actions from compact convex sets and receive only bandit feedback. Our main contribution is the design of an uncoupled learning algorithm that guarantees last-iterate convergence to the Nash equilibrium with high probability. We establish a convergence rate of $\tilde{O}(T^{-1/4})$ up to polynomial factors in problem parameters. Crucially, our proposed algorithm is computationally efficient, requiring only an efficient linear optimization oracle over the players' compact action sets. The algorithm is obtained by combining techniques from experimental design and the classic Follow-The-Regularized-Leader (FTRL) framework, with a carefully chosen regularizer function tailored to the geometry of the action set of each learner.

Theorems & Definitions (16)

• Lemma 2.1: Estimator concentration bound
• Lemma 2.2: chandrasekaran2012convex
• Lemma 2.3: Strong-convexity
• proof
• Proposition 2.4: Bregman divergence
• proof
• Theorem 2.5
• Lemma 3.0: RVU Property syrgkanis2015fast
• Lemma 3.0: RVU with estimation error
• Lemma A.1