On the price of exact truthfulness in incentive-compatible online learning with bandit feedback: A regret lower bound for WSU-UX
Ali Mortazavi, Junhao Lin, Nishant A. Mehta
TL;DR
The paper investigates incentive-compatible online learning with reputation-seeking experts under bandit feedback, focusing on the WSU-UX algorithm. Building on prior work, it shows a worst-case lower bound of $\Omega(T^{2/3})$ on regret for WSU-UX under valid hyperparameters, via a carefully constructed two-phase loss sequence with two arms. The result clarifies a gap between WSU-UX’s performance and the minimax rate, and suggests that improving beyond $O(T^{2/3})$ may require a fundamentally different IC approach. The analysis combines a potential-based comparison to EXP3 with a second-order lower-bound argument, leveraging martingale techniques to control the evolution of the probability distribution and the estimation errors. Overall, the work advances understanding of the intrinsic difficulty of truthful reporting in bandit online learning and delineates the limitations of a natural IC algorithm in this setting.
Abstract
In one view of the classical game of prediction with expert advice with binary outcomes, in each round, each expert maintains an adversarially chosen belief and honestly reports this belief. We consider a recently introduced, strategic variant of this problem with selfish (reputation-seeking) experts, where each expert strategically reports in order to maximize their expected future reputation based on their belief. In this work, our goal is to design an algorithm for the selfish experts problem that is incentive-compatible (IC, or \emph{truthful}), meaning each expert's best strategy is to report truthfully, while also ensuring the algorithm enjoys sublinear regret with respect to the expert with the best belief. Freeman et al. (2020) recently studied this problem in the full information and bandit settings and obtained truthful, no-regret algorithms by leveraging prior work on wagering mechanisms. While their results under full information match the minimax rate for the classical ("honest experts") problem, the best-known regret for their bandit algorithm WSU-UX is $O(T^{2/3})$, which does not match the minimax rate for the classical ("honest bandits") setting. It was unclear whether the higher regret was an artifact of their analysis or a limitation of WSU-UX. We show, via explicit construction of loss sequences, that the algorithm suffers a worst-case $Ω(T^{2/3})$ lower bound. Left open is the possibility that a different IC algorithm obtains $O(\sqrt{T})$ regret. Yet, WSU-UX was a natural choice for such an algorithm owing to the limited design room for IC algorithms in this setting.