Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fixed Budget is No Harder Than Fixed Confidence in Best-Arm Identification up to Logarithmic Factors

Kapilan Balagopalan, Yinan Li, Yao Zhao, Tuan Nguyen, Anton Daitche, Houssam Nassif, Kwang-Sung Jun

TL;DR

It is shown that FB is no harder than FC up to logarithmic factors, and this result not only reveals a fundamental relationship between FB and FC, but also has a significant implication: FC2FB, combined with existing state-of-the-art FC algorithms, leads to improved sample complexity for a number of FB problems.

Abstract

The best-arm identification (BAI) problem is one of the most fundamental problems in interactive machine learning, which has two flavors: the fixed-budget setting (FB) and the fixed-confidence setting (FC). For $K$-armed bandits with the unique best arm, the optimal sample complexities for both settings have been settled down, and they match up to logarithmic factors. This prompts an interesting research question about the generic, potentially structured BAI problems: Is FB harder than FC or the other way around? In this paper, we show that FB is no harder than FC up to logarithmic factors. We do this constructively: we propose a novel algorithm called FC2FB (fixed confidence to fixed budget), which is a meta algorithm that takes in an FC algorithm $\mathcal{A}$ and turn it into an FB algorithm. We prove that this FC2FB enjoys a sample complexity that matches, up to logarithmic factors, that of the sample complexity of $\mathcal{A}$. This means that the optimal FC sample complexity is an upper bound of the optimal FB sample complexity up to logarithmic factors. Our result not only reveals a fundamental relationship between FB and FC, but also has a significant implication: FC2FB, combined with existing state-of-the-art FC algorithms, leads to improved sample complexity for a number of FB problems.

Fixed Budget is No Harder Than Fixed Confidence in Best-Arm Identification up to Logarithmic Factors

TL;DR

It is shown that FB is no harder than FC up to logarithmic factors, and this result not only reveals a fundamental relationship between FB and FC, but also has a significant implication: FC2FB, combined with existing state-of-the-art FC algorithms, leads to improved sample complexity for a number of FB problems.

Abstract

The best-arm identification (BAI) problem is one of the most fundamental problems in interactive machine learning, which has two flavors: the fixed-budget setting (FB) and the fixed-confidence setting (FC). For -armed bandits with the unique best arm, the optimal sample complexities for both settings have been settled down, and they match up to logarithmic factors. This prompts an interesting research question about the generic, potentially structured BAI problems: Is FB harder than FC or the other way around? In this paper, we show that FB is no harder than FC up to logarithmic factors. We do this constructively: we propose a novel algorithm called FC2FB (fixed confidence to fixed budget), which is a meta algorithm that takes in an FC algorithm and turn it into an FB algorithm. We prove that this FC2FB enjoys a sample complexity that matches, up to logarithmic factors, that of the sample complexity of . This means that the optimal FC sample complexity is an upper bound of the optimal FB sample complexity up to logarithmic factors. Our result not only reveals a fundamental relationship between FB and FC, but also has a significant implication: FC2FB, combined with existing state-of-the-art FC algorithms, leads to improved sample complexity for a number of FB problems.
Paper Structure (26 sections, 25 theorems, 120 equations, 5 figures, 2 tables, 6 algorithms)

This paper contains 26 sections, 25 theorems, 120 equations, 5 figures, 2 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. From Fixed Confidence to Fixed Budget
  4. Warm up: A naive framework for known $A$ and $C$:
  5. The FC2FB algorithm.
  6. From Weak to Strong Fixed-Confidence Algorithms
  7. Applications
  8. Known Heterogeneous Noise
  9. Heterogeneous Noise with Unknown Variance
  10. Linear Bandits
  11. Unimodal Bandits
  12. Experiments
  13. Conclusion
  14. Appendix
  15. Related Work
  16. ...and 11 more sections

Key Result

Proposition 2.1

Suppose that an FB algorithm satisfies $B \ge B_0 \implies \mathop{\mathrm{\mathbb{P}}}\nolimits(\hat{J} \neq 1) \le F\exp(- \frac{B}{H})$. Then, there exists $B' = \Theta(H\ln(F/\delta) + B_0)$ such that if $B \ge B'$ then $\mathop{\mathrm{\mathbb{P}}}\nolimits(\hat{J} \neq 1) \le \delta$.

Figures (5)

  • Figure 1: Probability of misidentifying the best arm in the Gaussian bandit as a function of the budget $B$, with $K = 32$ arms. Results are averaged over $5{,}000$ trials.
  • Figure 2: Probability of misidentifying the best arm in the Gaussian bandit as a function of the number of arms. The budget is fixed at $6000$. Results are averaged over $5{,}000$ trials.
  • Figure 3: Execution routine of Algorithm \ref{['alg:fc2fb']}
  • Figure 4: Probability of misidentifying the best arm in the Gaussian bandit as a function of the budget $B$, with $K = 32$ arms. Results are averaged over $5{,}000$ trials.
  • Figure 5: Probability of misidentifying the best arm in the Gaussian bandit as a function of the number of arms. The budget is fixed at $6000$. Results are averaged over $5{,}000$ trials.

Theorems & Definitions (52)

  • Proposition 2.1
  • proof
  • Definition 3.1: Strong FC algorithm
  • Theorem 3.2: Correctness of FC2FB
  • Definition 4.1: Weak Fixed-Confidence Algorithm
  • Proposition 4.2: Correctness of $FCW2S(\delta)$
  • Proposition 4.3: Strong stopping time guarantee of $FCW2S(\delta)$
  • Theorem 5.1: The sample complexity of Algorithm \ref{['alg_main_FCESR']}
  • Corollary 5.2: Error probability bound for FC2FB(PE-KHN)
  • Corollary 5.3: Error probability bound for FC2FB(VD-BESTARMID)
  • ...and 42 more