Merit-based Fair Combinatorial Semi-Bandit with Unrestricted Feedback Delays
Ziqun Chen, Kechao Cai, Zhuoyue Chen, Jinbei Zhang, John C. S. Lui
TL;DR
This work studies stochastic combinatorial multi-armed bandits with unrestricted feedback delays under merit-based fairness constraints, motivated by crowdsourcing and online advertising. It introduces two delay models—reward-independent and reward-dependent—and proves that the optimal fair policy induces arm-selection probabilities proportional to a merit function: $p_a^* = \dfrac{L f(\mu_a)}{\sum_{a'} f(\mu_{a'})}$. The authors design fair CMAB algorithms (FCUCB-D, FCTS-D, OP-FCUCB-D, OP-FCTS-D) tailored to each delay setting and establish sublinear expected reward and fairness regrets, with regret bounds depending on delay quantiles $d^*(q)$. Across extensive synthetic and real-data experiments, the methods achieve merit-based fairness while maintaining favorable reward performance, even under heavy-tailed or infinite-delay regimes. The results demonstrate that fairness-aware, delay-tolerant bandits can effectively balance exploration/exploitation with equitable arm treatment in practical applications.
Abstract
We study the stochastic combinatorial semi-bandit problem with unrestricted feedback delays under merit-based fairness constraints. This is motivated by applications such as crowdsourcing, and online advertising, where immediate feedback is not immediately available and fairness among different choices (or arms) is crucial. We consider two types of unrestricted feedback delays: reward-independent delays where the feedback delays are independent of the rewards, and reward-dependent delays where the feedback delays are correlated with the rewards. Furthermore, we introduce merit-based fairness constraints to ensure a fair selection of the arms. We define the reward regret and the fairness regret and present new bandit algorithms to select arms under unrestricted feedback delays based on their merits. We prove that our algorithms all achieve sublinear expected reward regret and expected fairness regret, with a dependence on the quantiles of the delay distribution. We also conduct extensive experiments using synthetic and real-world data and show that our algorithms can fairly select arms with different feedback delays.