Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Series Expansion of Probability of Correct Selection for Improved Finite Budget Allocation in Ranking and Selection

Xinbo Shi, Yijie Peng, Bruno Tuffin

Abstract

This paper addresses the challenge of improving finite sample performance in Ranking and Selection by developing a Bahadur-Rao type expansion for the Probability of Correct Selection (PCS). While traditional large deviations approximations captures PCS behavior in the asymptotic regime, they can lack precision in finite sample settings. Our approach enhances PCS approximation under limited simulation budgets, providing more accurate characterization of optimal sampling ratios and optimality conditions dependent of budgets. Algorithmically, we propose a novel finite budget allocation (FCBA) policy, which sequentially estimates the optimality conditions and accordingly balances the sampling ratios. We illustrate numerically on toy examples that our FCBA policy achieves superior PCS performance compared to tested traditional methods. As an extension, we note that the non-monotonic PCS behavior described in the literature for low-confidence scenarios can be attributed to the negligence of simultaneous incorrect binary comparisons in PCS approximations. We provide a refined expansion and a tailored allocation strategy to handle low-confidence scenarios, addressing the non-monotonicity issue.

Series Expansion of Probability of Correct Selection for Improved Finite Budget Allocation in Ranking and Selection

Abstract

This paper addresses the challenge of improving finite sample performance in Ranking and Selection by developing a Bahadur-Rao type expansion for the Probability of Correct Selection (PCS). While traditional large deviations approximations captures PCS behavior in the asymptotic regime, they can lack precision in finite sample settings. Our approach enhances PCS approximation under limited simulation budgets, providing more accurate characterization of optimal sampling ratios and optimality conditions dependent of budgets. Algorithmically, we propose a novel finite budget allocation (FCBA) policy, which sequentially estimates the optimality conditions and accordingly balances the sampling ratios. We illustrate numerically on toy examples that our FCBA policy achieves superior PCS performance compared to tested traditional methods. As an extension, we note that the non-monotonic PCS behavior described in the literature for low-confidence scenarios can be attributed to the negligence of simultaneous incorrect binary comparisons in PCS approximations. We provide a refined expansion and a tailored allocation strategy to handle low-confidence scenarios, addressing the non-monotonicity issue.

Paper Structure

This paper contains 22 sections, 14 theorems, 141 equations, 3 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Setting and Main Results
  3. Notations.
  4. Ranking and Selection.
  5. The Bahadur-Rao Type Expansion.
  6. Approximation Methodology
  7. Large Deviations of Sample Mean
  8. Probability of Incorrect Binary Comparison
  9. Probability of Incorrect Multiple Comparison
  10. Proof of Theorem \ref{['thm:main']}
  11. Allocation Policy
  12. Optimality Conditions
  13. Experiments
  14. FCBA($\ell$) versus OCBA
  15. Finite Sample Performances
  16. ...and 7 more sections

Key Result

Theorem 1

Under Assumptions ass:light and ass:btv, for any $\ell\in\mathbb{N}$ and $p_{i}>0$, $\forall i\in[k]$, satisfying $\sum_{i\in[k]}p_{i} = 1$, we have the following expansion

Figures (3)

  • Figure 1: Left: PCS of FCBA($\ell$) versus OCBA under varying number of total simulation budget based on $5,000$ macro-replications. Right: Theoretically optimal allocation based on $V_{0}(\bm{p})$.
  • Figure 2: PCS before the budget is exhausted estimated by $100,000$ macro-replications in an instance with means in the stepping and with variances in the equal setting.
  • Figure 3: Left: PCS of LC-FCBA(0) versus compared algorithms based on $1,000,000$ macro-replications. Right: Average sample allocation when the best alternative is correctly identified.

Theorems & Definitions (33)

  • Theorem 1: Main
  • Example 1: Gaussian
  • Example 2: Exponential
  • Lemma 1
  • Proposition 1
  • Proposition 2
  • proof
  • Lemma 2
  • Proposition 3
  • proof
  • ...and 23 more