Multi-view Bayesian optimisation in an input-output reduced space for engineering design

TL;DR

The paper tackles the scalability of Bayesian optimisation for high-dimensional engineering design by introducing a probabilistic, multi-view latent space learned from both inputs and outputs via probabilistic partial least squares (PPLS). By embedding BO in the latent space and marginalising over latent variables, the approach yields a GP surrogate that better captures the intrinsic dimensionality of design problems and enhances exploration. Across illustrative and engineering-scale examples, PPLS-BO demonstrates faster convergence and robustness to latent-dimension misspecification compared with deterministic PLS-BO, PCA-BO, and classical BO. The work presents a practical framework for efficient design optimisation in complex FE-based problems, with potential extensions to non-smooth objectives and alternative surrogates.

Abstract

Bayesian optimisation is an adaptive sampling strategy for constructing a Gaussian process surrogate to efficiently search for the global minimum of a black-box computational model. Gaussian processes have limited applicability in engineering design problems, which usually have many design variables but typically a low intrinsic dimensionality. Their scalability can be significantly improved by identifying a low-dimensional space of latent variables that serve as inputs to the Gaussian process. In this paper, we introduce a multi-view learning strategy that considers both the input design variables and output data representing the objective or constraint functions, to identify a low-dimensional latent subspace. Adopting a fully probabilistic viewpoint, we use probabilistic partial least squares (PPLS) to learn an orthogonal mapping from the design variables to the latent variables using training data consisting of inputs and outputs of the black-box computational model. The latent variables and posterior probability densities of the PPLS and Gaussian process models are determined sequentially and iteratively, with retraining occurring at each adaptive sampling iteration. We compare the proposed probabilistic partial least squares Bayesian optimisation (PPLS-BO) strategy with its deterministic counterpart, partial least squares Bayesian optimisation (PLS-BO), and classical Bayesian optimisation, demonstrating significant improvements in convergence to the global minimum.

Figures (15)

• Figure 1: Graphical model of PPLS where $n$ is the size of the training data set $\mathcal{D}$, $\vec{s}$ are the design variables, $\vec{z}$ are the low-dimensional latent variables, $\vec{y}$ are the observations, and $\vec{\zeta}$ and $\vec{\upsilon}$ are two vectors containing the PPLS model hyperparameters.
• Figure 2: Graphical model of GP regression where $n$ is the size of the training data set $\mathcal{D}$, $\vec{z}$ are the latent variables (GP inputs), $f$ is the target output variable, $y$ is the observation, and $\vec{\theta} \cup \{\sigma_y\}$ are the model hyperparameters.
• Figure 3: Illustrative one-dimensional example for the GP regression in reduced dimension. Two realisations of the GP posterior density $p_{\Theta}(\vec{f}_*\,\vert\,\vec{y},\vec{z}, \vec{z}_*)$ and $p_{\Theta'}(\vec{f}_*\,\vert\,\vec{y},\vec{z}', \vec{z}_*)$ as in \ref{['eq:gp_5']} are fitted to training data $\mathcal{D}$ and $\mathcal{D'}$ composed of the same observation vector $\vec{y}$ but different (iid) latent variables sampled from their posterior densities $\vec{z}, \vec{z}' \sim \prod_{i=1}^n p_{\Phi}(z_i\vert y_i,\vec{s}_i)$ given by \ref{['eq:ppls-bo_2']}. The marginal posterior predictive density $p_{\Theta,\Phi}\left(\vec{f}_* \vert \vec{y}, \vec{S}\right)$ from \ref{['eq:ppls-bo_1']} is trained by marginalising over the training $\vec{z}$ and test $\vec{z}_*$ latent variables. In PPLS-BO, the acquisition function $a_{ei}(\cdot)$ is fitted to the mean $\hat{\mu}(\bar{\vec{z}}_*)$ and standard deviation $\hat{\sigma}(\bar{\vec{z}}_*)$ obtained from the marginal predictive posterior, and a new sample $\bar{z}_{k+1}$ is proposed.
• Figure 4: Graphical model for the $k$th adaptive sampling iteration of PPLS-BO with a training data set $\mathcal{D}$ of size $n+k$ with (random) design variables $\vec{s}$ and observations $\vec{y}=(y^{(1)}\,\,y^{(2)}\,\,...\,\,y^{(d_y)})$. Unobserved latent variables $\vec{z}$ and hyperparameters $\vec{\zeta} = \{\vec{W},\hat{\vec{\Sigma}}_s\}$ and $\vec{\upsilon} = \{\vec{Q},\hat{\vec{\Sigma}}_y\}$ are learned using PPLS independent from the GP surrogates with target output variables $\vec{f}=(f^{(1)}\,\,f^{(2)}\,\,...\,\,f^{(d_y)})$ and GP hyperparameters $\{\vec{\theta}^{(1)},\vec{\theta}^{(2)},...,\vec{\theta}^{(d_y)}\}$ and $\{\sigma_y^{(1)},\sigma_y^{(2)},...,\sigma_y^{(d_y)}\}$.
• Figure 5: Schematic of the mean latent variable domain $D_{\bar{z}}$ composed of the union between the feasible subdomain $D_{\bar{z}}^f$ contained within the design variable domain $D_s$, and the infeasible subdomain $D_{\bar{z}}^{nf}$ which extends beyond the design variable domain, where $\vec{S}$ are the design variables and $\vec{S}_e$ are the extreme design variables found at the vertices of the hyperplane describing the design variable domain.