Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning with a Budget: Identifying the Best Arm with Resource Constraints

Zitian Li, Wang Chi Cheung

TL;DR

The SH-RR algorithm is proposed, which integrates resource-aware allocation into the classical successive halving framework on best arm identification and unifies the theoretical analysis for both the stochastic and deterministic consumption settings.

Abstract

In many applications, evaluating the effectiveness of different alternatives comes with varying costs or resource usage. Motivated by such heterogeneity, we study the Best Arm Identification with Resource Constraints (BAIwRC) problem, where an agent seeks to identify the best alternative (aka arm) in the presence of resource constraints. Each arm pull consumes one or more types of limited resources. We make two key contributions. First, we propose the Successive Halving with Resource Rationing (SH-RR) algorithm, which integrates resource-aware allocation into the classical successive halving framework on best arm identification. The SH-RR algorithm unifies the theoretical analysis for both the stochastic and deterministic consumption settings, with a new \textit{effective consumption measure

Learning with a Budget: Identifying the Best Arm with Resource Constraints

TL;DR

The SH-RR algorithm is proposed, which integrates resource-aware allocation into the classical successive halving framework on best arm identification and unifies the theoretical analysis for both the stochastic and deterministic consumption settings.

Abstract

In many applications, evaluating the effectiveness of different alternatives comes with varying costs or resource usage. Motivated by such heterogeneity, we study the Best Arm Identification with Resource Constraints (BAIwRC) problem, where an agent seeks to identify the best alternative (aka arm) in the presence of resource constraints. Each arm pull consumes one or more types of limited resources. We make two key contributions. First, we propose the Successive Halving with Resource Rationing (SH-RR) algorithm, which integrates resource-aware allocation into the classical successive halving framework on best arm identification. The SH-RR algorithm unifies the theoretical analysis for both the stochastic and deterministic consumption settings, with a new \textit{effective consumption measure
Paper Structure (24 sections, 9 theorems, 96 equations, 2 figures, 2 tables, 1 algorithm)

This paper contains 24 sections, 9 theorems, 96 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Main Contributions
  3. Literature Review
  4. Problem Formulation
  5. Algorithm
  6. Performance Guarantees of SH-RR
  7. Lower Bound of $\Pr(\psi \neq 1)$
  8. Lower Bound for General Consumption Setting
  9. Lower Bound for Bernoulli Consumption Setting
  10. Numeric Experiments
  11. Conclusion
  12. Auxiliary Results
  13. Proofs
  14. Proof of Claim \ref{['claim:feasibility']}
  15. Proof of Theorem \ref{['theorem:upper-bound-of-failure']} and Theorem \ref{['theorem:upper-bound-of-failure-det']}
  16. ...and 9 more sections

Key Result

Theorem 1

Consider a BAIwRC instance $Q$. The SH-RR algorithm has BAI failure probability $\Pr(\psi \neq 1)$ at most where $\gamma(Q) = \min_{\ell\in [L]}\{C_\ell / H_{2, \ell}(Q)\}$, and $H_{2, \ell}(Q)$ is defined in (eq:sto_H).

Figures (2)

  • Figure 1: Convergence rates of $\log(\Pr(\psi \neq 1))$, with $10^7$ repeated trials
  • Figure 2: Comparison of SH-RR and anytime baselines in different setups

Theorems & Definitions (16)

  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Lemma 4: Bennett
  • Corollary 5
  • proof
  • proof
  • ...and 6 more