Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Player Resource-Sharing Games with Fair Reward Allocation

Mevan Wijewardena, Michael. J Neely

TL;DR

This work addresses worst-case learning for online, multi-player resource-sharing games with fair reward allocation and bandit feedback. It introduces a novel UCB-based algorithm that uses Madow's sampling to realize probabilistic resource selections and projects updates onto the $(n,r)$-hypersimplex, achieving a worst-case suboptimality of $\mathcal{O}(\log T/\sqrt{T})$ without knowledge of the horizon. When the mean rewards $\boldsymbol{E}$ are known, the paper derives explicit maximin solutions for key cases ($m=2,r=1$ and $m=3,r=1$) and shows how to reduce the general problem to tractable linear programs; it also provides structural results to identify $\boldsymbol{p}^*$ efficiently. Simulations demonstrate fast convergence of the learned policy to the optimal worst-case value and highlight clear behavioral patterns in resource selection as $E_k$ vary. The results contribute practical worst-case guarantees for online decentralized resource sharing, with relevance to MAC, network design, and congestion control.

Abstract

This paper considers an online multi-player resource-sharing game with bandit feedback. Multiple players choose from a finite collection of resources in a time slotted system. In each time slot, each resource brings a random reward that is equally divided among the players who choose it. The reward vector is independent and identically distributed over the time slots. The statistics of the reward vector are unknown to the players. During each time slot, for each resource chosen by the first player, they receive as feedback the reward of the resource and the number of players who chose it, after the choice is made. We develop a novel Upper Confidence Bound (UCB) algorithm that learns the mean rewards using the feedback and maximizes the worst-case time-average expected reward of the first player. The algorithm gets within $\mathcal{O}(\log(T)/\sqrt{T})$ of optimality within $T$ time slots. The simulations depict fast convergence of the learnt policy in comparison to the worst-case optimal policy.

Multi-Player Resource-Sharing Games with Fair Reward Allocation

TL;DR

This work addresses worst-case learning for online, multi-player resource-sharing games with fair reward allocation and bandit feedback. It introduces a novel UCB-based algorithm that uses Madow's sampling to realize probabilistic resource selections and projects updates onto the -hypersimplex, achieving a worst-case suboptimality of without knowledge of the horizon. When the mean rewards are known, the paper derives explicit maximin solutions for key cases ( and ) and shows how to reduce the general problem to tractable linear programs; it also provides structural results to identify efficiently. Simulations demonstrate fast convergence of the learned policy to the optimal worst-case value and highlight clear behavioral patterns in resource selection as vary. The results contribute practical worst-case guarantees for online decentralized resource sharing, with relevance to MAC, network design, and congestion control.

Abstract

This paper considers an online multi-player resource-sharing game with bandit feedback. Multiple players choose from a finite collection of resources in a time slotted system. In each time slot, each resource brings a random reward that is equally divided among the players who choose it. The reward vector is independent and identically distributed over the time slots. The statistics of the reward vector are unknown to the players. During each time slot, for each resource chosen by the first player, they receive as feedback the reward of the resource and the number of players who chose it, after the choice is made. We develop a novel Upper Confidence Bound (UCB) algorithm that learns the mean rewards using the feedback and maximizes the worst-case time-average expected reward of the first player. The algorithm gets within of optimality within time slots. The simulations depict fast convergence of the learnt policy in comparison to the worst-case optimal policy.
Paper Structure (24 sections, 23 theorems, 233 equations, 4 figures, 4 algorithms)

This paper contains 24 sections, 23 theorems, 233 equations, 4 figures, 4 algorithms.

Table of Contents

  1. Introduction
  2. Time Average Expected Reward
  3. Worst-Case Time Average Expected Reward
  4. Related work
  5. Background on Resource-Sharing Games
  6. Contributions
  7. Notation
  8. Bandit Algorithm
  9. Analysis of the Algorithm
  10. Finding $f^{\text{worst},*}$ with known $\boldsymbol{E}$
  11. $r=1$, $m=2$
  12. $r=1$, $m=3$
  13. Solving (P1-1)
  14. Solving (P1-2)
  15. Finding $\boldsymbol{p}^*$
  16. ...and 9 more sections

Key Result

Lemma 1

Given a sequence $\{X_t\}_{t=1}^{\infty}$ of independent zero-mean $1$-sub Gaussian random variables, a positive integer-valued random variable $G$ (possibly dependent on the sequence $\{X_t\}_{t=1}^{\infty}$) and $\epsilon \in (0,1)$, we have

Figures (4)

  • Figure 1: Left: The mean rewards of the resources, Right: Probabilities of choosing the resources
  • Figure 2: Left: The mean rewards of different resources. Right: Probabilities of choosing different resources for the considered $\boldsymbol{E}$.
  • Figure 3: Scenario $\boldsymbol{E} =[3,1,1,1,0.5,0.1]$. Top:$\frac{1}{t}\sum_{\tau=1}^tf^{\text{worst}}(\boldsymbol{p}(\tau))$ and $f^{\text{worst},*}$ vs $t$, Bottom: Components of $\boldsymbol{p}(t)$ and components of $\boldsymbol{p}^*$ vs $t$ for , Left:$r = 1$, Middle:$r = 2$, Right:$r = 3$
  • Figure 4: Scenario $\boldsymbol{E} = [6.1,1,1,1,0.5,0.1]$. Top:$\frac{1}{t}\sum_{\tau=1}^tf^{\text{worst}}(\boldsymbol{p}(\tau))$ and $f^{\text{worst},*}$ vs $t$, Bottom: Components of $\boldsymbol{p}(t)$ and components of $\boldsymbol{p}^*$ vs $t$, Left:$r = 1$, Middle:$r = 2$, Right:$r = 3$

Theorems & Definitions (54)

  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Theorem 1
  • proof
  • Lemma 3
  • proof
  • Lemma 4
  • proof
  • ...and 44 more