Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Regret Analysis of Sleeping Competing Bandits

Shinnosuke Uba, Yutaro Yamaguchi

Abstract

The Competing Bandits framework is a recently emerging area that integrates multi-armed bandits in online learning with stable matching in game theory. While conventional models assume that all players and arms are constantly available, in real-world problems, their availability can vary arbitrarily over time. In this paper, we formulate this setting as Sleeping Competing Bandits. To analyze this problem, we naturally extend the regret definition used in existing competing bandits and derive regret bounds for the proposed model. We propose an algorithm that simultaneously achieves an asymptotic regret bound of $\mathrm{O}\left(NK\log T_{i}/Δ^2\right)$ under reasonable assumptions, where $N$ is the number of players, $K$ is the number of arms, $T_{i}$ is the number of rounds of each player $p_i$, and $Δ$ is the minimum reward gap. We also provide a regret lower bound of $\mathrmΩ\left( N(K-N+1)\log T_{i}/Δ^2 \right)$ under the same assumptions. This implies that our algorithm is asymptotically optimal in the regime where the number of arms $K$ is relatively larger than the number of players $N$.

Regret Analysis of Sleeping Competing Bandits

Abstract

The Competing Bandits framework is a recently emerging area that integrates multi-armed bandits in online learning with stable matching in game theory. While conventional models assume that all players and arms are constantly available, in real-world problems, their availability can vary arbitrarily over time. In this paper, we formulate this setting as Sleeping Competing Bandits. To analyze this problem, we naturally extend the regret definition used in existing competing bandits and derive regret bounds for the proposed model. We propose an algorithm that simultaneously achieves an asymptotic regret bound of under reasonable assumptions, where is the number of players, is the number of arms, is the number of rounds of each player , and is the minimum reward gap. We also provide a regret lower bound of under the same assumptions. This implies that our algorithm is asymptotically optimal in the regime where the number of arms is relatively larger than the number of players .
Paper Structure (31 sections, 13 theorems, 60 equations, 3 figures, 3 algorithms)

This paper contains 31 sections, 13 theorems, 60 equations, 3 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Main Contributions
  3. Related Work
  4. Single Player Multi-Armed Bandit
  5. Multi-Player Bandits
  6. Competing Bandits
  7. Proposed Model and Methods
  8. Basic Notation and Definitions
  9. Definitions of Regret
  10. Proposed Methods
  11. UCB and LCB
  12. Awake Centralized UCB Algorithm
  13. Awake Centralized Explore-Then-Gale--Shapley Algorithm
  14. Regret Lower Bounds
  15. Regret Upper Bounds
  16. ...and 16 more sections

Key Result

theorem 1

For any policy and any constant $c \in (0, 1)$, there exists a problem instance (a collection of reward distributions) such that the player-optimal stable regret $\overline{R}_i(T_i)$ and player-pessimal stable regret $\underline{R}_i(T_i)$ for some player $p_i$ satisfies:

Figures (3)

  • Figure 1: Regret comparison between random and weighted exploration with heterogeneous player unavailability probabilities.
  • Figure 2: Regret comparison between random and weighted exploration with identical player unavailability probabilities.
  • Figure 3: Regret comparison between random and weighted exploration with identical player unavailability probabilities, where the preferences of arms are fixed.

Theorems & Definitions (30)

  • definition 1: Stable Matching gale1962college
  • definition 2: Player-Optimal/Pessimal Stable Regrets
  • definition 3: $\alpha$-Consistency salomon2013lower
  • theorem 1
  • proof
  • theorem 2
  • proof
  • theorem 3
  • proof
  • definition 4: Blocking Triplet (Extended)
  • ...and 20 more