Large-Batch, Iteration-Efficient Neural Bayesian Design Optimization

TL;DR

A highly scalable, sample-based acquisition function that performs a non-dominated sorting of not only the objectives but also their associated uncertainty is presented, which shows that the acquisition function in combination with different Bayesian neural network surrogates is effective in data-intensive environments with a minimal number of iterations.

Abstract

Bayesian optimization (BO) provides a powerful framework for optimizing black-box, expensive-to-evaluate functions. It is therefore an attractive tool for engineering design problems, typically involving multiple objectives. Thanks to the rapid advances in fabrication and measurement methods as well as parallel computing infrastructure, querying many design problems can be heavily parallelized. This class of problems challenges BO with an unprecedented setup where it has to deal with very large batches, shifting its focus from sample efficiency to iteration efficiency. We present a novel Bayesian optimization framework specifically tailored to address these limitations. Our key contribution is a highly scalable, sample-based acquisition function that performs a non-dominated sorting of not only the objectives but also their associated uncertainty. We show that our acquisition function in combination with different Bayesian neural network surrogates is effective in data-intensive environments with a minimal number of iterations. We demonstrate the superiority of our method by comparing it with state-of-the-art multi-objective optimizations. We perform our evaluation on two real-world problems -- airfoil design and 3D printing -- showcasing the applicability and efficiency of our approach. Our code is available at: https://github.com/an-on-ym-ous/lbn_mobo

Figures (22)

• Figure 1: Optimizing 6D ZDT3 problem using a range of acquisition functions and surrogates.
• Figure 2: LBN-MOBO starts with training a Bayesian neural network ($f_{BNN}$) on random designs. We then run our acquisition function ($A\text{\tiny F}$) and compute a 2MD Pareto front to explore promising (green) and under-represented regions (red) of the NFP. We then append the acquired candidates to the data set and retrain $f_{BNN}$. By incorporating uncertainty information alongside the Pareto front of the best performances (blue candidates), we identify promising candidates in areas of high uncertainty, where there is potential for additional information (red candidates).
• Figure 3: ZDT3 benchmark optimization via 2MD acquisition function and various neural BO surrogate models. The fitting time of each model was recorded to assess computational efficiency.
• Figure 4: We present the evolution of hypervolume and the Pareto front for both the airfoil and the printer color gamut problems, utilizing LBN-MOBO with DE and MC dropout as the surrogate models. Figure (a) shows the hypervolume expansion of the Airfoil problem, and (b) represents the Pareto front calculated using each surrogate. Similarly, (c) depicts the hypervolume expansion of the printer color gamut problem, and (d) displays the gamut actually discovered by LBN-MOBO using both surrogates.
• Figure 5: Ablation studies on the effect of epistemic uncertainty in our $2\textit{M}D$ acquisition function, using our real-world problems.