On the development of a practical Bayesian optimisation algorithm for expensive experiments and simulations with changing environmental conditions

Abstract

Experiments in engineering are typically conducted in controlled environments where parameters can be set to any desired value. This assumes that the same applies in a real-world setting -- an assumption that is often incorrect as many experiments are influenced by uncontrollable environmental conditions such as temperature, humidity and wind speed. When optimising such experiments, the focus should lie on finding optimal values conditionally on these uncontrollable variables. This article extends Bayesian optimisation to the optimisation of systems in changing environments that include controllable and uncontrollable parameters. The extension fits a global surrogate model over all controllable and environmental variables but optimises only the controllable parameters conditional on measurements of the uncontrollable variables. The method is validated on two synthetic test functions and the effects of the noise level, the number of the environmental parameters, the parameter fluctuation, the variability of the uncontrollable parameters, and the effective domain size are investigated. ENVBO, the proposed algorithm resulting from this investigation, is applied to a wind farm simulator with eight controllable and one environmental parameter. ENVBO finds solutions for the full domain of the environmental variable that outperforms results from optimisation algorithms that only focus on a fixed environmental value in all but one case while using a fraction of their evaluation budget. This makes the proposed approach very sample-efficient and cost-effective. An off-the-shelf open-source version of ENVBO is available via the NUBO Python package.

Figures (10)

• Figure 1: Bayesian optimisation applied to a 1-dimensional function with one local and one global maximum. Expected improvement is used as the acquisition function. The input space is bounded by $[0, 10]$
• Figure 2: Maximisation of a two-dimensional problem with one environmental variable $x_1$ and one controllable variable $x_2$. Yellow areas indicate high outputs and dark blue areas indicate low outputs. Upper-left: True objective function. Upper-right: Prediction of a Gaussian process with a measurement taken for the next conditional optimisation step. Lower-left: Gaussian process prediction for optimisation conditional on the measurement. Lower-right: Bayesian optimisation step conditional on the measurement
• Figure 3: Two-dimensional negated Levy function with one controllable parameter $x_1$ bounded by $[-7.5, 7.5]$ and one uncontrollable variable $x_2$ bounded by $[-10, 10]$
• Figure 4: Upper row: Means (lines) and $95$% confidence intervals (shaded areas) of the mean absolute percentage error between Gaussian process prediction and truth over $30$ replications. Lower row: Difference between algorithms and random benchmark after $100$ function evaluations for each of the $30$ replications. Two-dimensional Levy function with one uncontrollable parameter on the left and six-dimensional Hartmann function with one uncontrollable parameter on the right
• Figure 5: Comparison of different trade-off parameters $\beta$ for the upper confidence bound acquisition function. Means (lines) and $95$% confidence intervals (shaded areas) of the mean absolute percentage error between Gaussian process prediction and truth over $30$ replications. Two-dimensional Levy function with one uncontrollable parameter on the left and six-dimensional Hartmann function with one uncontrollable parameter on the right