Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adjusted Expected Improvement for Cumulative Regret Minimization in Noisy Bayesian Optimization

Shouri Hu, Haowei Wang, Zhongxiang Dai, Bryan Kian Hsiang Low, Szu Hui Ng

TL;DR

This work addresses the inefficiency of EI under cumulative regret in noisy Bayesian optimization by introducing the evaluation-cost (EIC) algorithm. EIC augments EI with a data-informed downside cost, leading to a budget-aware sampling rule and a provably tight high-probability regret bound $R_N = O(\sqrt{N}\gamma_N(\log N)^{1/2})$ that matches GP-UCB/GP-TS benchmarks. Theoretical results are complemented by synthetic and real-world experiments showing EIC consistently reduces cumulative regret relative to EI, EI-Nyugen, GP-UCB, and GP-TS. The approach offers a principled balance between exploration and exploitation and suggests directions for extending BO to richer domains and incorporating prior knowledge.

Abstract

The expected improvement (EI) is one of the most popular acquisition functions for Bayesian optimization (BO) and has demonstrated good empirical performances in many applications for the minimization of simple regret. However, under the evaluation metric of cumulative regret, the performance of EI may not be competitive, and its existing theoretical regret upper bound still has room for improvement. To adapt the EI for better performance under cumulative regret, we introduce a novel quantity called the evaluation cost which is compared against the acquisition function, and with this, develop the expected improvement-cost (EIC) algorithm. In each iteration of EIC, a new point with the largest acquisition function value is sampled, only if that value exceeds its evaluation cost. If none meets this criteria, the current best point is resampled. This evaluation cost quantifies the potential downside of sampling a point, which is important under the cumulative regret metric as the objective function value in every iteration affects the performance measure. We establish in theory a high-probability regret upper bound of EIC based on the maximum information gain, which is tighter than the bound of existing EI-based algorithms. It is also comparable to the regret bound of other popular BO algorithms such as Thompson sampling (GP-TS) and upper confidence bound (GP-UCB). We further perform experiments to illustrate the improvement of EIC over several popular BO algorithms.

Adjusted Expected Improvement for Cumulative Regret Minimization in Noisy Bayesian Optimization

TL;DR

This work addresses the inefficiency of EI under cumulative regret in noisy Bayesian optimization by introducing the evaluation-cost (EIC) algorithm. EIC augments EI with a data-informed downside cost, leading to a budget-aware sampling rule and a provably tight high-probability regret bound that matches GP-UCB/GP-TS benchmarks. Theoretical results are complemented by synthetic and real-world experiments showing EIC consistently reduces cumulative regret relative to EI, EI-Nyugen, GP-UCB, and GP-TS. The approach offers a principled balance between exploration and exploitation and suggests directions for extending BO to richer domains and incorporating prior knowledge.

Abstract

The expected improvement (EI) is one of the most popular acquisition functions for Bayesian optimization (BO) and has demonstrated good empirical performances in many applications for the minimization of simple regret. However, under the evaluation metric of cumulative regret, the performance of EI may not be competitive, and its existing theoretical regret upper bound still has room for improvement. To adapt the EI for better performance under cumulative regret, we introduce a novel quantity called the evaluation cost which is compared against the acquisition function, and with this, develop the expected improvement-cost (EIC) algorithm. In each iteration of EIC, a new point with the largest acquisition function value is sampled, only if that value exceeds its evaluation cost. If none meets this criteria, the current best point is resampled. This evaluation cost quantifies the potential downside of sampling a point, which is important under the cumulative regret metric as the objective function value in every iteration affects the performance measure. We establish in theory a high-probability regret upper bound of EIC based on the maximum information gain, which is tighter than the bound of existing EI-based algorithms. It is also comparable to the regret bound of other popular BO algorithms such as Thompson sampling (GP-TS) and upper confidence bound (GP-UCB). We further perform experiments to illustrate the improvement of EIC over several popular BO algorithms.
Paper Structure (19 sections, 9 theorems, 62 equations, 3 figures, 1 table, 1 algorithm)

This paper contains 19 sections, 9 theorems, 62 equations, 3 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Statement and Background
  3. Gaussian Processes
  4. Expected Improvement
  5. Related Works
  6. The Expected Improvement-Cost (EIC) algorithm
  7. Initial Experiment Scheme
  8. Acquisition Function
  9. Evaluation Cost and the EIC Algorithm
  10. Intuitions behind the Evaluation Cost.
  11. Regret Analysis of EIC
  12. Reproducing Kernel Hilbert Space
  13. Regret Upper Bound of EIC
  14. Regret Upper Bound
  15. Proof of Theorem \ref{['thm1']}
  16. ...and 4 more sections

Key Result

Theorem 1

Assume (A1)-- (A3). Let $0 < \delta <1$. With probability at least $1-\delta$, running EIC algorithm under a GP model with prior mean function $\mu({\bf x}) \equiv 0$, and with parameters (initdesign), (incum), (omgn), $\lambda^2 = 1 + 2/N$ achieves

Figures (3)

  • Figure 1: Cumulative regret of common BO methods and our proposed EIC algorithm on the Ackley test function. The solid line represents the cumulative regret (averaged over 100 independent runs) and the shaded area is the corresponding 95% confidence region.
  • Figure 2: Cumulative regret of different BO algorithms on six test functions.
  • Figure 3: Cumulative regret of different BO algorithms in the neural network hyper-parameter tuning experiment.

Theorems & Definitions (10)

  • Theorem 1
  • Remark 1
  • Corollary 1
  • Lemma 1: Theorem 2 of CG17
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • Lemma 7