Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multiple-play Stochastic Bandits with Prioritized Arm Capacity Sharing

Hong Xie, Haoran Gu, Yanying Huang, Tao Tan, Defu Lian

TL;DR

This work introduces MSB-PRS, a principled variant of multi-play stochastic bandits that models prioritized resource sharing with stochastic arm capacity. It develops a computationally efficient oracle based on a priority-aware bipartite matching formulation and an approximate UCB learning algorithm that achieves near-optimal regret: instance-independent bounds with a $\sqrt{K\ln KT}$ factor and instance-dependent bounds with a $α_1 K^2$ factor. Theoretical contributions include instance-independent and instance-dependent regret lower bounds and matching upper bounds for the proposed ApUCB, despite the nonlinear combinatorial utility induced by the priority-sharing mechanism. Empirical results on synthetic data demonstrate sublinear regret and superior performance of MSB-PRS-ApUCB over baselines across varying numbers of arms, plays, movement costs, and reward variances, suggesting practicality for differentiated-service resource allocation in LLM and edge-computing contexts.

Abstract

This paper proposes a variant of multiple-play stochastic bandits tailored to resource allocation problems arising from LLM applications, edge intelligence, etc. The model is composed of $M$ arms and $K$ plays. Each arm has a stochastic number of capacities, and each unit of capacity is associated with a reward function. Each play is associated with a priority weight. When multiple plays compete for the arm capacity, the arm capacity is allocated in a larger priority weight first manner. Instance independent and instance dependent regret lower bounds of $Ω( α_1 σ\sqrt{KM T} )$ and $Ω(α_1 σ^2 \frac{M}Δ \ln T)$ are proved, where $α_1$ is the largest priority weight and $σ$ characterizes the reward tail. When model parameters are given, we design an algorithm named \texttt{MSB-PRS-OffOpt} to locate the optimal play allocation policy with a computational complexity of $O(MK^3)$. Utilizing \texttt{MSB-PRS-OffOpt} as a subroutine, an approximate upper confidence bound (UCB) based algorithm is designed, which has instance independent and instance dependent regret upper bounds matching the corresponding lower bound up to factors of $ \sqrt{K \ln KT }$ and $α_1 K^2$ respectively. To this end, we address nontrivial technical challenges arising from optimizing and learning under a special nonlinear combinatorial utility function induced by the prioritized resource sharing mechanism.

Multiple-play Stochastic Bandits with Prioritized Arm Capacity Sharing

TL;DR

This work introduces MSB-PRS, a principled variant of multi-play stochastic bandits that models prioritized resource sharing with stochastic arm capacity. It develops a computationally efficient oracle based on a priority-aware bipartite matching formulation and an approximate UCB learning algorithm that achieves near-optimal regret: instance-independent bounds with a factor and instance-dependent bounds with a factor. Theoretical contributions include instance-independent and instance-dependent regret lower bounds and matching upper bounds for the proposed ApUCB, despite the nonlinear combinatorial utility induced by the priority-sharing mechanism. Empirical results on synthetic data demonstrate sublinear regret and superior performance of MSB-PRS-ApUCB over baselines across varying numbers of arms, plays, movement costs, and reward variances, suggesting practicality for differentiated-service resource allocation in LLM and edge-computing contexts.

Abstract

This paper proposes a variant of multiple-play stochastic bandits tailored to resource allocation problems arising from LLM applications, edge intelligence, etc. The model is composed of arms and plays. Each arm has a stochastic number of capacities, and each unit of capacity is associated with a reward function. Each play is associated with a priority weight. When multiple plays compete for the arm capacity, the arm capacity is allocated in a larger priority weight first manner. Instance independent and instance dependent regret lower bounds of and are proved, where is the largest priority weight and characterizes the reward tail. When model parameters are given, we design an algorithm named \texttt{MSB-PRS-OffOpt} to locate the optimal play allocation policy with a computational complexity of . Utilizing \texttt{MSB-PRS-OffOpt} as a subroutine, an approximate upper confidence bound (UCB) based algorithm is designed, which has instance independent and instance dependent regret upper bounds matching the corresponding lower bound up to factors of and respectively. To this end, we address nontrivial technical challenges arising from optimizing and learning under a special nonlinear combinatorial utility function induced by the prioritized resource sharing mechanism.
Paper Structure (21 sections, 7 theorems, 89 equations, 4 figures, 2 algorithms)

This paper contains 21 sections, 7 theorems, 89 equations, 4 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. MSB-PRS Model
  5. Model Setting
  6. Problem Formulation
  7. Fundamental Learning Limits
  8. Efficient Learning Algorithms
  9. Efficient Computation Oracle
  10. Efficient Learning Algorithm
  11. Synthetic Experiments
  12. Parameter setting
  13. Performance Evaluation
  14. Conclusion
  15. Proof of Lemma \ref{['lem:offline:ActionToMatching']}
  16. ...and 6 more sections

Key Result

Theorem 1

For any learning algorithm, there exists an instance of MSB-PRS such that Furthermore, $\text{Reg}_T \geq \Omega(\alpha_1 \sigma \sqrt{MK T})$.

Figures (4)

  • Figure 1: Impact of Number of Arms.
  • Figure 2: Impact of Number of plays.
  • Figure 3: Impact of Movement Cost.
  • Figure 4: Impact of Standard Deviation of Reward.

Theorems & Definitions (9)

  • Theorem 1
  • Theorem 2
  • Definition 3
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • Theorem 7
  • Theorem 8
  • Definition 9