Sample Complexity of Identifying the Nonredundancy of Nontransitive Games in Dueling Bandits
Shang Lu, Shuji Kijima
TL;DR
This work studies the problem of identifying nonredundancy in nontransitive two-player zero-sum games represented by a skew-symmetric payoff matrix $A\in[-1,1]^{n\times n}$ with odd $n$. It builds a dueling-bandit framework where duel outcomes are $1$-sub-Gaussian with mean $a_{ij}$ and independent across pairs, and develops an $(\alpha,\delta)$-PAC algorithm whose sample complexity scales as $O\left(\dfrac{\varphi(A)^2}{\max\{\alpha^2,\pi_{\min}^2\}} \log \dfrac{n}{\delta}\right)$, where $\varphi(A)$ captures a stability/property of $A$ via $\det(A_j)$ and $\pi$, the Nash equilibrium solving $\mathbf{x}^T A=0^T$. The paper also proves lower bounds of $\Omega\left(\dfrac{1}{\alpha^2}\log \dfrac{1}{\delta}\right)$ and $\Omega\left(\varphi(A)^2 \log \dfrac{1}{\delta}\right)$ for certain $n$, implying a gap between upper and lower bounds and highlighting instance-dependent complexity. By tying nonredundancy to completely mixed Nash equilibria and Pfaffian conditions, the results illuminate how spectral/structural properties of $A$ govern learnability in nontransitive settings. This advances understanding of learning in nontransitive dueling environments and informs design of experiments and algorithms for identifying indispensable moves in complex strategic games.
Abstract
Dueling bandit is a variant of the Multi-armed bandit to learn the binary relation by comparisons. Most work on the dueling bandit has targeted transitive relations, that is, totally/partially ordered sets, or assumed at least the existence of a champion such as Condorcet winner and Copeland winner. This work develops an analysis of dueling bandits for non-transitive relations. Jan-ken (a.k.a. rock-paper-scissors) is a typical example of a non-transitive relation. It is known that a rational player chooses one of three items uniformly at random, which is known to be Nash equilibrium in game theory. Interestingly, any variant of Jan-ken with four items (e.g., rock, paper, scissors, and well) contains at least one useless item, which is never selected by a rational player. This work investigates a dueling bandit problem to identify whether all $n$ items are indispensable in a given win-lose relation. Then, we provide upper and lower bounds of the sample complexity of the identification problem in terms of the determinant of $A$ and a solution of $\mathbf{x}^{\top} A = \mathbf{0}^{\top}$ where $A$ is an $n \times n$ pay-off matrix that every duel follows.