On the Implementation of a Bayesian Optimization Framework for Interconnected Systems

TL;DR

This work tackles efficient optimization of grey-box, interconnected systems by extending Bayesian optimization to composite objectives $f(x, y(x))$ where $y(x)$ is learned via Gaussian processes. The proposed BOIS framework implements adaptive linearization to derive closed-form analytical moments for $f$, enabling a local Gaussian approximation through a Laplace-like scheme and reducing surrogate complexity. Across chemical-process and photobioreactor case studies, BOIS consistently matches or outperforms standard BO and existing composite-function BO methods while delivering substantial reductions in computational cost, especially compared with Monte Carlo approaches. The results demonstrate the practical value of structure-exploiting, feasible-aware surrogate modeling for large-scale, physics-informed design under uncertainty, with clear avenues for extending to higher dimensions and different probabilistic models.

Abstract

Bayesian optimization (BO) is an effective paradigm for the optimization of expensive-to-sample systems. Standard BO learns the performance of a system $f(x)$ by using a Gaussian Process (GP) model; this treats the system as a black-box and limits its ability to exploit available structural knowledge (e.g., physics and sparse interconnections in a complex system). Grey-box modeling, wherein the performance function is treated as a composition of known and unknown intermediate functions $f(x, y(x))$ (where $y(x)$ is a GP model) offers a solution to this limitation; however, generating an analytical probability density for $f$ from the Gaussian density of $y(x)$ is often an intractable problem (e.g., when $f$ is nonlinear). Previous work has handled this issue by using sampling techniques or by solving an auxiliary problem over an augmented space where the values of $y(x)$ are constrained by confidence intervals derived from the GP models; such solutions are computationally intensive. In this work, we provide a detailed implementation of a recently proposed grey-box BO paradigm, BOIS, that uses adaptive linearizations of $f$ to obtain analytical expressions for the statistical moments of the composite function. We show that the BOIS approach enables the exploitation of structural knowledge, such as that arising in interconnected systems as well as systems that embed multiple GP models and combinations of physics and GP models. We benchmark the effectiveness of BOIS against standard BO and existing grey-box BO algorithms using a pair of case studies focused on chemical process optimization and design. Our results indicate that BOIS performs as well as or better than existing grey-box methods, while also being less computationally intensive.

Figures (12)

• Figure 1: Grey-box systems often exhibit a known structure where the connectivity between different elements is understood. Not every component is always a black-box that requires a surrogate model as a closed-form representation might be available for various components.
• Figure 2: Illustration of the adaptive linearization scheme employed by BOIS. At a point $x$ of interest (red marker), a Gaussian process model estimates the value of intermediate $y$. A local Laplace approximation is then constructed by linearizing $f$ around a neighboring point (green marker). The summarizing statistics are passed into an acquisition function that determines the value of sampling at the selected point. This process is repeated until the optimum of the acquisition function is found.
• Figure 3: Workflow of the S-BO framework. Using a dataset $\mathcal{D}^{\ell}$, BO builds a GP surrogate model of the system. The mean and variance estimates calculated by the GP are passed into an acquisition function that is optimized to suggest a new sampling point $x^{\ell+1}$. The system is sampled at this point and the collected data is appended to the dataset to retrain the GP model.
• Figure 4: Workflow of the OP-BO algorithm. The data in $\mathcal{D}_y^{\ell}$ is used to construct $\mathcal{GP}_y^{\ell}$. The mean and uncertainty estimates calculated by the surrogate model are used to create a confidence interval bounded by $l_y^{\ell}(x)$ and $u_y^{\ell}(x)$ that constrains the possible values of $y$. These are incorporated into an auxiliary problem that is optimized to select a new sample point $x^{\ell+1}$. The resulting data is then appended to the dataset and the GP models are retrained
• Figure 5: Schematic representation of a nested function structure for $y$.