Exploring Exploration in Bayesian Optimization

TL;DR

This work addresses the lack of a quantitative exploration metric in Bayesian optimization by introducing two measures, Observation Traveling Salesman Distance ($OTSD$) and Observation Entropy ($OE$), that quantify how AFs explore the observation space. The authors derive $OTSD$ and $OE$ (with bounds) and demonstrate their usefulness across a wide range of synthetic and real-world benchmarks, including high-dimensional problems, while showing how batching and acceleration techniques affect exploration. The study yields an empirical taxonomy of AF exploration, revealing that top-performing methods balance exploration and exploitation rather than maximizing either extreme, and provides practical guidance for AF design and portfolio construction. By offering non-parametric, problem-aware metrics and bounding properties, this work enables principled analysis and steering of AFs in BO, including extensions to non-Euclidean domains and future AF development.

Abstract

A well-balanced exploration-exploitation trade-off is crucial for successful acquisition functions in Bayesian optimization. However, there is a lack of quantitative measures for exploration, making it difficult to analyze and compare different acquisition functions. This work introduces two novel approaches - observation traveling salesman distance and observation entropy - to quantify the exploration characteristics of acquisition functions based on their selected observations. Using these measures, we examine the explorative nature of several well-known acquisition functions across a diverse set of black-box problems, uncover links between exploration and empirical performance, and reveal new relationships among existing acquisition functions. Beyond enabling a deeper understanding of acquisition functions, these measures also provide a foundation for guiding their design in a more principled and systematic manner.

Figures (34)

• Figure 1: Observations of UCB with various $\beta$ values ($1$, $10$, and $100$) on a two-dimensional GP-prior sample reveal the explorative behavior for different $\beta$. The black crosses are initial points, the orange plus signs are the observations of the BO phase, and the red star is the optimal location.
• Figure 2: The exploration quantities OTSD and OE of RS, UCB ($\beta=0.1$), EI, PI, MES, and deterministic selection (DM) on the 6-dimensional Hartmann function. From left to right, these plots show OTSD, normalized OTSD, and OE, respectively. DM values for OE (around $-134$) are hidden for better visualization. The shaded areas show the standard error of the mean.
• Figure 3: Normalized OTSD and OE rank averaged across all the synthetic benchmarks. A lower rank means lower exploration. We do not show the initial DoE phase.
• Figure 4: Normalized OTSD, average rank for OE and optimization performance on the synthetic benchmarks.
• Figure 5: Normalized OTSD, average ranks for OE and optimization performance for EI and its variations on the synthetic benchmarks.

Theorems & Definitions (5)

• Definition 3.1: Exploration
• Proposition 3.2: Upper and Lower Bounds for the True OTSD
• Proposition 3.3: Upper Bound for the True OE
• proof
• proof