Revisiting Social Welfare in Bandits: UCB is (Nearly) All You Need
Dhruv Sarkar, Nishant Pandey, Sayak Ray Chowdhury
TL;DR
The paper tackles fairness in sequential decision-making by studying Nash regret and its generalization to p-mean regret in stochastic bandits. It introduces Welfarist-UCB, a two-phase algorithm that couples a data-adaptive uniform exploration phase with a standard UCB exploration-exploitation phase, and proves near-optimal regret bounds under sub-Gaussian reward distributions without restrictive assumptions. The key contributions include showing that additive Hoeffding bounds suffice (avoiding multiplicative bounds), achieving order-optimal Nash regret NR_T = O(\tilde{O}(\sigma sqrt{ k log T log k / T })) and extending the framework to all p-mean regrets with corresponding bounds; and demonstrating a no-free-lunch phenomenon as fairness becomes more stringent (p → −∞). The results have practical implications for fairness-aware online decision-making in settings like clinical trials and public resource allocation, where equitable outcomes across rounds are crucial.
Abstract
Regret in stochastic multi-armed bandits traditionally measures the difference between the highest reward and either the arithmetic mean of accumulated rewards or the final reward. These conventional metrics often fail to address fairness among agents receiving rewards, particularly in settings where rewards are distributed across a population, such as patients in clinical trials. To address this, a recent body of work has introduced Nash regret, which evaluates performance via the geometric mean of accumulated rewards, aligning with the Nash social welfare function known for satisfying fairness axioms. To minimize Nash regret, existing approaches require specialized algorithm designs and strong assumptions, such as multiplicative concentration inequalities and bounded, non-negative rewards, making them unsuitable for even Gaussian reward distributions. We demonstrate that an initial uniform exploration phase followed by a standard Upper Confidence Bound (UCB) algorithm achieves near-optimal Nash regret, while relying only on additive Hoeffding bounds, and naturally extending to sub-Gaussian rewards. Furthermore, we generalize the algorithm to a broad class of fairness metrics called the $p$-mean regret, proving (nearly) optimal regret bounds uniformly across all $p$ values. This is in contrast to prior work, which made extremely restrictive assumptions on the bandit instances and even then achieved suboptimal regret bounds.