A Bi-Objective Optimization Based Acquisition Strategy for Batch Bayesian Global Optimization
Francesco Carciaghi, Simone Magistri, Pierluigi Mansueto, Fabio Schoen
TL;DR
The paper tackles batch Bayesian optimization for expensive black-box functions by reframing the acquisition step as a bi-objective problem on the GP posterior, using the pair $(\mu_k(x),-\sigma_k^2(x))$ to reconstruct the Pareto front of the BOO problem. It introduces an adaptation of the Non-dominated Sorting Memetic Algorithm (NSMA) to explicitly recover a rich set of exploitation–exploration trade-offs without predefined scalarization, and then employs two clustering strategies to select $q$ Pareto-front points for evaluation. Across extensive experiments on high- and low-dimensional benchmarks, NSMA-based BOO variants outperform traditional scalarized approaches (q-EI, q-LCB) and even NSGA-II in many high-dimensional settings, with domain-space clustering proving particularly effective when $n$ is large. The results demonstrate that Pareto-front reconstruction provides robust, scalable acquisition decisions that reduce the number of expensive evaluations while maintaining strong convergence behavior, making the approach attractive for parallelized Bayes-Opt in complex design spaces.
Abstract
In this paper, we deal with batch Bayesian Optimization (Bayes-Opt) problems over a box and we propose a novel bi-objective optimization (BOO) acquisition strategy to sample points where to evaluate the objective function. The BOO problem involves the Gaussian Process posterior mean and variance functions, which, in most of the acquisition strategies from the literature, are generally used in combination, frequently through scalarization. However, such scalarization could compromise the Bayes-Opt process performance, as getting the desired trade-off between exploration and exploitation is not trivial in most cases. We instead aim to reconstruct the Pareto front of the BOO problem based on optimizing both the posterior mean as well as the variance, thus generating multiple trade-offs without any a priori knowledge. The reconstruction is performed through the Non-dominated Sorting Memetic Algorithm (NSMA), recently proposed in the literature and proved to be effective in solving hard MOO problems. Finally, we present two clustering approaches, each of them operating on a different space, to select potentially optimal points from the Pareto front. We compare our methodology with well-known acquisition strategies from the literature, showing its effectiveness on a wide set of experiments.