Generative Multi-Form Bayesian Optimization

TL;DR

The paper tackles the challenge of optimizing expensive black-box objectives over structured inputs by introducing Generative Multi-Form Optimization (GMFoO). GMFoO jointly explores one high-dimensional latent space and multiple low-dimensional latent spaces using MFoO-GAN to induce positive correlations and enable cross-space knowledge transfer, guided by Bayesian optimization with multi-fidelity surrogates. The key contributions are the GMFoO framework, the Info-GAN-inspired correlation mechanism, and the enhanced local exploitation plus multi-fidelity GP instantiation, which together improve convergence speed and final design quality. Empirical results on airfoil and corbel design problems and a MNIST-based area maximization task demonstrate faster and more accurate optimization under limited budgets, highlighting the practical value of multi-form latent-space optimization for complex, high-dimensional design problems.

Abstract

Many real-world problems, such as airfoil design, involve optimizing a black-box expensive objective function over complex structured input space (e.g., discrete space or non-Euclidean space). By mapping the complex structured input space into a latent space of dozens of variables, a two-stage procedure labeled as generative model based optimization (GMO) in this paper, shows promise in solving such problems. However, the latent dimension of GMO is hard to determine, which may trigger the conflicting issue between desirable solution accuracy and convergence rate. To address the above issue, we propose a multi-form GMO approach, namely generative multi-form optimization (GMFoO), which conducts optimization over multiple latent spaces simultaneously to complement each other. More specifically, we devise a generative model which promotes positive correlation between latent spaces to facilitate effective knowledge transfer in GMFoO. And further, by using Bayesian optimization (BO) as the optimizer, we propose two strategies to exchange information between these latent spaces continuously. Experimental results are presented on airfoil and corbel design problems and an area maximization problem as well to demonstrate that our proposed GMFoO converges to better designs on a limited computational budget.

Figures (18)

• Figure 1: Standard process of generative model based optimization (GMO)
• Figure 2: GMFoO framework, where the generative model is labeled as MFoO-GAN
• Figure 3: Sample synthesis and exchange process between the high- and lower-dimensional latent space, (a) in the low-dimensional latent space, (b) in the high-dimensional latent space
• Figure 4: Enhanced local exploitation of the high-dimensional latent space by leveraging information of the optimal solution of the low-dimensional latent space
• Figure 5: Leveraging knowledge through the multi-fidelity modeling, (a) in the high-dimensional latent space, (b) in the low-dimensional latent space