Weighted Wasserstein Barycenter of Gaussian Processes for exotic Bayesian Optimization tasks

TL;DR

This work introduces Weighted Wasserstein Barycenter of Gaussian Processes (W2BGP) as a unifying approach to exotic Bayesian Optimization tasks, including collaborative/federated BO, batch BO, and multi-fidelity BO, by tuning a simple weighting scheme over GP posteriors. It leverages the fact that GP predictions are Gaussian at each input, enabling a tractable univariate Gaussian WB and allowing standard acquisition functions such as LCB, PI, and EI to be reformulated under the WB framework. The paper examines multiple weighting strategies across tasks, showing that self-confident weighting often yields the best performance for collaboration, while uncooperative weighting promotes batch diversity, and fidelities as weights are not universally superior for MFBO. Empirical results on diverse test problems demonstrate the framework’s ability to unify these tasks under a single mechanism, with acquisition interpretation and computational gains highlighted. The work also points to future directions including transfer-BO, scalable GPs, and adaptive weighting schemes, and provides open-source code and data for reproducibility.

Abstract

Exploiting the analogy between Gaussian Distributions and Gaussian Processes' posterior, we present how the weighted Wasserstein Barycenter of Gaussian Processes (W2BGP) can be used to unify, under a common framework, different exotic Bayesian Optimization (BO) tasks. Specifically, collaborative/federated BO, (synchronous) batch BO, and multi-fidelity BO are considered in this paper. Our empirical analysis proves that each one of these tasks requires just an appropriate weighting schema for the W2BGP, while the entire framework remains untouched. Moreover, we demonstrate that the most well-known BO acquisition functions can be easily re-interpreted under the proposed framework and also enable a more computationally efficient way to deal with the computation of the Wasserstein Barycenter, compared with state-of-the-art methods from the Machine Learning literature. Finally, research perspectives branching from the proposed approach are presented.

Figures (18)

• Figure 1: Two GPs fitted on the same set of observations. At $x=0.81$ is it possible to observe: the normal distribution associated to the first GP, namely $\mathcal{N}(\mu_1(0.81),\sigma_1(0.81))$ (blue solid curve), normal distribution associated to the second GP, namely $\mathcal{N}(\mu_2(0.81),\sigma_2(0.81))$ (green solid curve), and the resulting WB, with $\lambda_1=\lambda_2=0.5$ (dashed purple curve).
• Figure 2: The resulting WB of the two GPs in Figure \ref{['fig:1']}, with $\lambda_1=\lambda_2=0.5$.
• Figure 3: The three weighting scheme for the W2BGP in collaborative/federated BO: self-confident weighting schema (up-left), equally-weighted schema (up-right), and uncooperative weighting schema (bottom).
• Figure 4: Graphical representation of the self-confident (left) and uncooperative (right) weighting schemes for batch BO. Equally-weighted schema is just model averaging and it is omitted in the figure.
• Figure 5: Collaborative/Federated BO: gap metric curves (median and standard deviation on 30 independent runs) of the three weighting scheme on the Alpine01 test problem and for different dimensionality of the search space ($d=\{5,10,20\}$.