Voronoi Candidates for Bayesian Optimization

TL;DR

Bayesian optimization for expensive black-box functions is often bottlenecked by the costly acquisition subproblem. The authors introduce Vorcands, a discrete, data-adaptive candidate set drawn from the boundary $oundary\mathcal{V}_{\mathcal{X}}$ of the Voronoi tessellation of current design points, implemented via Vorwalk with implicit sampling or projection. Across high-dimensional benchmarks with a $GP$ surrogate and $EI$ acquisitions, Vorcands deliver substantial reductions in wall-clock time while maintaining or improving the best observed value, outperforming continuous optimization and common candidate schemes in many cases. The approach is scalable, parallelizable, and applicable to a broader class of surrogates and acquisitions, with potential extensions to batch, multi-objective, or constrained settings and future work exploring alternative sampling strategies and metrics.

Abstract

Bayesian optimization (BO) offers an elegant approach for efficiently optimizing black-box functions. However, acquisition criteria demand their own challenging inner-optimization, which can induce significant overhead. Many practical BO methods, particularly in high dimension, eschew a formal, continuous optimization of the acquisition function and instead search discretely over a finite set of space-filling candidates. Here, we propose to use candidates which lie on the boundary of the Voronoi tessellation of the current design points, so they are equidistant to two or more of them. We discuss strategies for efficient implementation by directly sampling the Voronoi boundary without explicitly generating the tessellation, thus accommodating large designs in high dimension. On a battery of test problems optimized via Gaussian processes with expected improvement, our proposed approach significantly improves the execution time of a multi-start continuous search without a loss in accuracy.

Figures (14)

• Figure 1: EI surface of a toy problem and its local optima with $\ell_\infty$ Voronoi tesselation (gray lines; see Section \ref{['sec:bg_vor']}) superimposed. Dotted gray line gives new boundary after adding the blue "+".
• Figure 2: Comparing candidates for BO in 2 dimensions. "x"s represent $\mathbf{X}$ while open circles represent candidates. Candidates in the same quadrant share a color. Tricands has a limited number of total candidates; all other figures show 1,000 candidates.
• Figure 3: Comparing candidates for BO in 10 dimensions, the first two are shown. "x"s represent $\mathbf{X}$ while open circles represent candidates. Candidates in the same orthant share a color, except those not sharing an orthant with any design point, which are gray. All figures show 1,000 candidates.
• Figure 4: Example Voronoi cells induced by the 1, 2 and $\infty$ norm.
• Figure 5: As shown in the left panel, any point on the boundary of a Voronoi cell (black x) can be represented with Cartesian coordinates per usual, but may also be parameterized by the angle one has to travel (in blue) to reach it if starting at the design point defining the cell (red x). The right panel illustrates a bisection search to resolve the Cartesian coordinates of the implied boundary point given the starting point and the search direction.