Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Best-Arm Identification in Unimodal Bandits

Riccardo Poiani, Marc Jourdan, Emilie Kaufmann, Rémy Degenne

TL;DR

This work addresses fixed-confidence best-arm identification in unimodal bandits, showing that unimodality induces sparsity in the optimal sampling allocation, with only the best arm and its neighbors driving the leading cost asymptotically. It introduces TaS- and Top-Two-based strategies that exploit the unimodal structure, providing asymptotic optimality for TaS variants in one-parameter exponential families and non-asymptotic, near-optimal guarantees for Gaussian settings in UniTT, plus efficient implementations. The analysis establishes both a favorable instance-dependent lower bound (sparsity) and a minimax lower bound (linear in $K$) to delineate regimes where structure helps. Empirically, the proposed methods show strong performance and scalability, validating their practical relevance for unimodal decision problems.

Abstract

We study the fixed-confidence best-arm identification problem in unimodal bandits, in which the means of the arms increase with the index of the arm up to their maximum, then decrease. We derive two lower bounds on the stopping time of any algorithm. The instance-dependent lower bound suggests that due to the unimodal structure, only three arms contribute to the leading confidence-dependent cost. However, a worst-case lower bound shows that a linear dependence on the number of arms is unavoidable in the confidence-independent cost. We propose modifications of Track-and-Stop and a Top Two algorithm that leverage the unimodal structure. Both versions of Track-and-Stop are asymptotically optimal for one-parameter exponential families. The Top Two algorithm is asymptotically near-optimal for Gaussian distributions and we prove a non-asymptotic guarantee matching the worse-case lower bound. The algorithms can be implemented efficiently and we demonstrate their competitive empirical performance.

Best-Arm Identification in Unimodal Bandits

TL;DR

This work addresses fixed-confidence best-arm identification in unimodal bandits, showing that unimodality induces sparsity in the optimal sampling allocation, with only the best arm and its neighbors driving the leading cost asymptotically. It introduces TaS- and Top-Two-based strategies that exploit the unimodal structure, providing asymptotic optimality for TaS variants in one-parameter exponential families and non-asymptotic, near-optimal guarantees for Gaussian settings in UniTT, plus efficient implementations. The analysis establishes both a favorable instance-dependent lower bound (sparsity) and a minimax lower bound (linear in ) to delineate regimes where structure helps. Empirically, the proposed methods show strong performance and scalability, validating their practical relevance for unimodal decision problems.

Abstract

We study the fixed-confidence best-arm identification problem in unimodal bandits, in which the means of the arms increase with the index of the arm up to their maximum, then decrease. We derive two lower bounds on the stopping time of any algorithm. The instance-dependent lower bound suggests that due to the unimodal structure, only three arms contribute to the leading confidence-dependent cost. However, a worst-case lower bound shows that a linear dependence on the number of arms is unavoidable in the confidence-independent cost. We propose modifications of Track-and-Stop and a Top Two algorithm that leverage the unimodal structure. Both versions of Track-and-Stop are asymptotically optimal for one-parameter exponential families. The Top Two algorithm is asymptotically near-optimal for Gaussian distributions and we prove a non-asymptotic guarantee matching the worse-case lower bound. The algorithms can be implemented efficiently and we demonstrate their competitive empirical performance.

Paper Structure

This paper contains 89 sections, 50 theorems, 234 equations, 11 figures, 1 table, 2 algorithms.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Contributions
  4. Setting and Notation
  5. LOWER BOUNDS
  6. Dependence in the Bandit Model $\mu$ and the Risk Parameter $\delta$
  7. Dependence in the Number of Arms
  8. ALGORITHMS
  9. Notation
  10. Recommendation and Stopping Rules
  11. Full-sum Stopping Rule
  12. Local GLR Stopping Rule
  13. Track-and-stop
  14. Asymptotic Optimality
  15. Efficient Implementation
  16. ...and 74 more sections

Key Result

Theorem 2.1

For any $\delta$-correct algorithm and any unimodal instance $\bm \nu \in \mathcal{D}^{K}$ with mean $\bm{\mu} \in \mathcal{S}$, we have $\mathbb{E}_{\bm{\nu}}[\tau_\delta] \ge T^\star(\bm{\mu}) \log \frac{1}{2.4 \delta}$ with where $\textup{Alt}(\bm{\mu}) = \{ \bm{\theta} \in \mathcal{S}: i^{\star}(\bm{\theta}) \ne i^{\star}(\bm{\mu}) \}$ and $\bm{\omega}^\star(\bm{\mu})$ denote the maximizer of

Figures (11)

  • Figure 1: Empirical stopping time ($\delta = 0.01$) on (top) random instances with $K \in \{10,100\}$, denoted as $\bm{\mu}_{\textup{R}, 10}$ and $\bm{\mu}_{\textup{R}, 100}$, and (bottom) flat instances with $K \in (11,101)$, denoted as $\bm{\mu}_{\textup{F}, 11}$ and $\bm{\mu}_{\textup{F}, 101}$.
  • Figure 2: Ratio of characteristic times $T^\star_{1/2}(\bm{\mu}) / T^{\star}(\bm{\mu})$ when $|\mathcal{N}(i^\star)|=2$ as a function of $x = (\mu_{i^\star} - \mu_{j})/(\mu_{i^\star} - \mu_{j^\star})$ where $j^\star \in \operatornamewithlimits{argmax}_{i \in \mathcal{N}(i^\star)} \mu_i$ and $j \in \mathcal{N}(i^\star)\setminus \{j^\star\}$. The dashed blue line is $r_{2} = 6/(1+\sqrt{2})^2$.
  • Figure 3: Ablation on $K$ in random instances. Empirical sample complexity on $\bm{\mu}_{\textup{R}, 10}$.
  • Figure 4: Ablation on $K$ in random instances. Empirical sample complexity on $\bm{\mu}_{\textup{R}, 100}$.
  • Figure 5: Ablation on $K$ in random instances. Empirical sample complexity on $\bm{\mu}_{\textup{R}, 500}$.
  • ...and 6 more figures

Theorems & Definitions (92)

  • Theorem 2.1
  • Theorem 2.2
  • Theorem 2.3
  • Lemma 3.1
  • Theorem 3.2
  • Theorem 3.3
  • Theorem 3.4
  • Theorem C.1
  • proof
  • Lemma C.0
  • ...and 82 more