Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Best Arm Identification with Resource Constraints

Zitian Li, Wang Chi Cheung

TL;DR

This work extends Best Arm Identification to resource-constrained, cost-heterogeneous experiments (BAIwRC), where each pull consumes multiple resource types and a fixed budget must be respected. It introduces SH-RR, a Sequential Halving variant with explicit resource rationing that ensures feasibility while maintaining exploration efficacy. The authors derive non-asymptotic upper bounds on the probability of failure, with distinct complexity terms for deterministic and stochastic consumption, and establish matching lower bounds to demonstrate near-optimality. They show that stochastic consumption induces fundamentally different, more challenging behavior than deterministic consumption, and validate the approach through extensive simulations and real-world hyperparameter tuning tasks. The results highlight the importance of accounting for resource costs in pure-exploration bandit problems and provide practical guidance for cost-aware experimentation.

Abstract

Motivated by the cost heterogeneity in experimentation across different alternatives, we study the Best Arm Identification with Resource Constraints (BAIwRC) problem. The agent aims to identify the best arm under resource constraints, where resources are consumed for each arm pull. We make two novel contributions. We design and analyze the Successive Halving with Resource Rationing algorithm (SH-RR). The SH-RR achieves a near-optimal non-asymptotic rate of convergence in terms of the probability of successively identifying an optimal arm. Interestingly, we identify a difference in convergence rates between the cases of deterministic and stochastic resource consumption.

Best Arm Identification with Resource Constraints

TL;DR

This work extends Best Arm Identification to resource-constrained, cost-heterogeneous experiments (BAIwRC), where each pull consumes multiple resource types and a fixed budget must be respected. It introduces SH-RR, a Sequential Halving variant with explicit resource rationing that ensures feasibility while maintaining exploration efficacy. The authors derive non-asymptotic upper bounds on the probability of failure, with distinct complexity terms for deterministic and stochastic consumption, and establish matching lower bounds to demonstrate near-optimality. They show that stochastic consumption induces fundamentally different, more challenging behavior than deterministic consumption, and validate the approach through extensive simulations and real-world hyperparameter tuning tasks. The results highlight the importance of accounting for resource costs in pure-exploration bandit problems and provide practical guidance for cost-aware experimentation.

Abstract

Motivated by the cost heterogeneity in experimentation across different alternatives, we study the Best Arm Identification with Resource Constraints (BAIwRC) problem. The agent aims to identify the best arm under resource constraints, where resources are consumed for each arm pull. We make two novel contributions. We design and analyze the Successive Halving with Resource Rationing algorithm (SH-RR). The SH-RR achieves a near-optimal non-asymptotic rate of convergence in terms of the probability of successively identifying an optimal arm. Interestingly, we identify a difference in convergence rates between the cases of deterministic and stochastic resource consumption.
Paper Structure (19 sections, 9 theorems, 88 equations, 2 figures, 2 tables, 1 algorithm)

This paper contains 19 sections, 9 theorems, 88 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Model
  3. The SH-RR Algorithm
  4. Performance Guarantees of SH-RR
  5. Lower Bounds to $\Pr(\psi\neq 1)$
  6. Numerical Experiments
  7. Auxiliary Results
  8. Proofs
  9. Proof of Claim \ref{['claim:feasibility']}
  10. Proof of Theorem \ref{['theorem:upper-bound-of-failure-det']}
  11. Proof of Theorem \ref{['theorem:upper-bound-of-failure']}
  12. Proof of Lemma \ref{['lemma:bound_count']}
  13. Proof of Theorem \ref{['theorem:lower-bound-fixed-consumption-multiple-resource']}
  14. Proof of Theorem \ref{['theorem:lower-bound-sto-consumption-multiple-resource']}
  15. Improvement of $H_{2, \ell}^{\text{det}}(Q)$ is Unachievable
  16. ...and 4 more sections

Key Result

Theorem 2

Consider a BAIwRC instance $Q$ in the deterministic consumption setting. SH-RR (Algorithm alg:Sequential-Halving) has BAI failure probability $\Pr(\psi \neq 1)$ at most where $\gamma^{\text{det}}(Q) = \min_{\ell\in [L]}\{C_\ell / H^{\text{det}}_{2, \ell}(Q)\}$, and $H^{\text{det}}_{2, \ell}(Q)$ is defined in (eq:det_H).

Figures (2)

  • Figure 1: Convergence rates of $\log(\Pr(\psi \neq 1))$, with $10^7$ repeated trials
  • Figure 2: Comparison of SH-RR and anytime baselines in different setups

Theorems & Definitions (12)

  • Claim 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • Lemma 6: Chernoff
  • proof : Proof of Claim \ref{['claim:feasibility']}
  • Lemma 7
  • Lemma 8: Lemma 4 in CarpentierL16
  • Lemma 9: csiszar1998method
  • ...and 2 more