Bayesian Optimization of Partially Known Systems using Hybrid Models

Abstract

Bayesian optimization (BO) has gained attention as an efficient algorithm for black-box optimization of expensive-to-evaluate systems, where the BO algorithm iteratively queries the system and suggests new trials based on a probabilistic model fitted to previous samples. Still, the standard BO loop may require a prohibitively large number of experiments to converge to the optimum, especially for high-dimensional and nonlinear systems. We present a hybrid model-based BO formulation that combines the iterative Bayesian learning of BO with partially known mechanistic physical models. Instead of learning a direct mapping from inputs to the objective, we write all known equations for a physics-based model and infer expressions for variables missing equations using a probabilistic model, in our case, a Gaussian process (GP). The final formulation then includes the GP as a constraint in the hybrid model, thereby allowing other physics-based nonlinear and implicit model constraints. This hybrid model formulation yields a constrained, nonlinear stochastic program, which we discretize using the sample-average approximation. In an in-silico optimization of a single-stage distillation, the hybrid BO model based on mass conservation laws yields significantly better designs than a standard BO loop. Furthermore, the hybrid model converges in as few as one iteration, depending on the initial samples, whereas, the standard BO does not converge within 25 for any of the seeds. Overall, the proposed hybrid BO scheme presents a promising optimization method for partially known systems, leveraging the strengths of both mechanistic modeling and data-driven optimization.

Figures (5)

• Figure 1: Log-regret $\mathrm{Regret} = f'-f^*$ of BO for the illustrative example for random sampling, standard BO with EI, and hybrid-BO with SAA-EI. The shaded areas represent the 90% intervals of the regret convergence over 25 runs.
• Figure 2: Progression of SAA-EI with hybrid acquisition problem \ref{['prob: illustrative example']}. The top row shows the GP approximation of the unknown equation $h(u)\approx\mathcal{GP}(u)$, the center row shows the approximation of the objective, and the bottom row shows the SAA-EI (Equation \ref{['eq: SAA EI']}). The approximations show the 90% confidence intervals, where the approximation of the objective is displayed as the quantiles estimated from 25 discrete samples $\xi_s$. Left: Iteration 1; center: Iteration 3; right: Iteration 6.
• Figure 3: Sketch of the flash unit with fixed inlet stream composition and temperature $T$ and pressure $p$ as the decision variables.
• Figure 4: Log regret (see Equation \ref{['eq: simple regret']}) hybrid BO in comparison to standard BO with EI and random sampling. Confidence intervals from eight runs of the SAA-EI and 25 random and standard BO.
• Figure 5: Comparison of the expected improvement (EI) acquisition functions in the Temperature - Pressure plane between the benchmark standard BO with EI (top) and the hybrid model BO using the SAA-EI function (bottom).