Tight Regret Upper and Lower Bounds for Optimistic Hedge in Two-Player Zero-Sum Games

TL;DR

This work analyzes learning dynamics based on optimistic Hedge in two-player zero-sum games, focusing on tight dependence on the numbers of actions $m$ and $n$ and on leading constants. By reformulating regret analysis as an optimization over learning rates and a negative-term tradeoff, it derives refined upper bounds, including a cardinality-aware $O(\sqrt{\log m \log n})$-type regime with explicit constants. It further provides algorithm-dependent lower bounds that match the social regret and, in many cases, the individual regrets, establishing optimality up to constants; the analysis extends to dynamic regret with improved last-iterate convergence and matching lower bounds. The results demonstrate that cardinality-awareness yields tangible performance gains and close gaps between upper and lower bounds, offering near-tight characterizations of regret and dynamic regret for optimistic Hedge in uncoupled learning in games. The findings have implications for equilibrium learning in large or asymmetric games and for designing fast-converging uncoupled learning dynamics in practice.

Abstract

In two-player zero-sum games, the learning dynamic based on optimistic Hedge achieves one of the best-known regret upper bounds among strongly-uncoupled learning dynamics. With an appropriately chosen learning rate, the social and individual regrets can be bounded by $O(\log(mn))$ in terms of the numbers of actions $m$ and $n$ of the two players. This study investigates the optimality of the dependence on $m$ and $n$ in the regret of optimistic Hedge. To this end, we begin by refining existing regret analysis and show that, in the strongly-uncoupled setting where the opponent's number of actions is known, both the social and individual regret bounds can be improved to $O(\sqrt{\log m \log n})$. In this analysis, we express the regret upper bound as an optimization problem with respect to the learning rates and the coefficients of certain negative terms, enabling refined analysis of the leading constants. We then show that the existing social regret bound as well as these new social and individual regret upper bounds cannot be further improved for optimistic Hedge by providing algorithm-dependent individual regret lower bounds. Importantly, these social regret upper and lower bounds match exactly including the constant factor in the leading term. Finally, building on these results, we improve the last-iterate convergence rate and the dynamic regret of a learning dynamic based on optimistic Hedge, and complement these bounds with algorithm-dependent dynamic regret lower bounds that match the improved bounds.

Figures (2)

• Figure 1: Tradeoff versus $\gamma\in(0,1)$ when $m = n = 10^2$: (a) $f^*(\gamma) = f(\lambda_\gamma)$ and $g^*(\gamma) = g(\lambda_\gamma)$ for $\lambda_{\gamma} = \mathop{\mathrm{arg\,min}}\limits_{\lambda\in\Lambda}J_{\gamma}(\lambda)$; (b) $J^{\ast}(\gamma) = J_\gamma(\lambda_\gamma)$ and $\max\{f^{\ast},g^{\ast}\}$.
• Figure 2: Regret versus the number of rounds for the learning dynamics based on the optimistic Hedge algorithm in the setting $m = 2$ and $n = 10^4$. From top-left to bottom-right: social regret, the maximum of the individual regrets, the regret of the $x$-player, and the regret of the $y$-player.

Theorems & Definitions (26)

• Theorem 1: freund99adaptive
• Lemma 2
• Theorem 3
• proof : Proof sketch of \ref{['thm:upper_bounds']}
• Lemma 4
• Theorem 5
• Theorem 6
• Lemma 7
• Theorem 8
• Theorem 9