High-Dimensional Bayesian Optimization Using Both Random and Supervised Embeddings

TL;DR

EGORSE introduces a high-dimensional Bayesian optimization framework that blends random and supervised linear embeddings to search large design spaces efficiently. By operating in a reduced subspace defined by transfer matrices, and by adaptively learning these subspaces with both PLS and MGP guidance, EGORSE reduces the burden of hyper-parameter estimation and acquisition optimization while maintaining connection to the full design space via backward mappings. The method shows strong performance on benchmark high-dimensional problems (5–600 variables), offering favorable CPU-time and sample-efficiency relative to state-of-the-art HDBO methods, and demonstrates practical applicability to aerospace-like planning problems. Limitations include dependence on an estimated effective dimension and potential subspace-induced bias, motivating future work on automatic dimension selection and constrained/guaranteed-convergence extensions.

Abstract

Bayesian optimization (BO) is one of the most powerful strategies to solve computationally expensive-to-evaluate blackbox optimization problems. However, BO methods are conventionally used for optimization problems of small dimension because of the curse of dimensionality. In this paper, a high-dimensionnal optimization method incorporating linear embedding subspaces of small dimension is proposed to efficiently perform the optimization. An adaptive learning strategy for these linear embeddings is carried out in conjunction with the optimization. The resulting BO method, named efficient global optimization coupled with random and supervised embedding (EGORSE), combines in an adaptive way both random and supervised linear embeddings. EGORSE has been compared to state-of-the-art algorithms and tested on academic examples with a number of design variables ranging from 10 to 600. The obtained results show the high potential of EGORSE to solve high-dimensional blackbox optimization problems, in terms of both CPU time and the limited number of calls to the expensive blackbox simulation.

Figures (9)

• Figure 1: Illustration of $\mathcal{B}^{(t)}$ and ${\mathcal{A}}^{(t)}$. In black, the points of $\mathcal{B}^{(t)}$ without image in $\Omega$ by $\gamma_B^{(t)}$; in white, the points of ${\mathcal{A}}^{(t)}$ corresponding to the points of $\mathcal{B}^{(t)}$ with an image in $\Omega$ by $\gamma_B^{(t)}$.
• Figure 2: Projection of an objective function of 10 design variables into a linear subspace of 2 dimensions and the corresponding ${\alpha^{(l)}_{EI_{ext}}}$ acquisition function. The grey area is the unfeasible domain, the green squares are DoE points of ${\mathcal{A}}^{(t)}$, the red squares are DoE points of $\mathcal{B}^{(t)} \backslash {\mathcal{A}}^{(t)}$ and the green star is a solution of the optimization sub-problem.
• Figure 3: An optimization problem \ref{['eq:it_pb']} in a 2 dimensional linear subspace and the associated SEGOMOE optimization sub-problem. The grey area is the unfeasible domain, the green squares are DoE points of ${\mathcal{A}}^{(t)}$, the red squares are DoE points of $\mathcal{B}^{(t)} \backslash {\mathcal{A}}^{(t)}$ and the green star is a solution of the optimization sub-problem.
• Figure 4: An XDSM diagram of the EGORSE framework.
• Figure 5: Convergence plots of 6 versions of EGORSE applied on the MB_10 problem. The grey vertical line shows the size of the initial DoE.