Bayesian Optimization of Function Networks with Partial Evaluations

TL;DR

This work addresses Bayesian optimization for objectives represented as a function network, where the final scalar $y_K(x)$ depends on intermediate node outputs. It introduces p-KGFN, a cost-aware knowledge-gradient acquisition that selects which node $k$ and input $z_k$ to evaluate next, enabling partial evaluations under a budget. The method uses independent Gaussian process priors for each node, fantasy sampling to propagate information to the final node, and MC/SAA-based procedures with a discretized inner optimization to scale acquisition maximization. Empirical results on synthetic networks and real-world problems (Ackley6D, Manu-GP, FreeSolv, Pharma) show p-KGFN consistently outperforming full-network baselines and other benchmarks, especially when downstream evaluations are costly. The approach is implemented in BoTorch and demonstrates substantial practical impact for cost-efficient, network-structured Bayesian optimization.

Abstract

Bayesian optimization is a powerful framework for optimizing functions that are expensive or time-consuming to evaluate. Recent work has considered Bayesian optimization of function networks (BOFN), where the objective function is given by a network of functions, each taking as input the output of previous nodes in the network as well as additional parameters. Leveraging this network structure has been shown to yield significant performance improvements. Existing BOFN algorithms for general-purpose networks evaluate the full network at each iteration. However, many real-world applications allow for evaluating nodes individually. To exploit this, we propose a novel knowledge gradient acquisition function that chooses which node and corresponding inputs to evaluate in a cost-aware manner, thereby reducing query costs by evaluating only on a part of the network at each step. We provide an efficient approach to optimizing our acquisition function and show that it outperforms existing BOFN methods and other benchmarks across several synthetic and real-world problems. Our acquisition function is the first to enable cost-aware optimization of a broad class of function networks.

Figures (15)

• Figure 1: An example function network in the manufacturing problem.
• Figure 2: Comparison of EIFN and p-KGFN on a 1-D synthetic two-stage function network $f_2(f_1(\cdot))$. Top row (left to right): Initial models for $f_1(\cdot)$, $f_2(\cdot)$ and $f_2(f_1(\cdot))$. Second and third rows: Resulting surrogate models upon budget depletion by EIFN and p-KGFN, respectively. Each ground truth function is represented by an orange curve, while blue curves and shaded blue areas denote posterior mean and standard deviation, respectively. Black stars indicate the initial three points fully evaluated across the network for both algorithms. Dark green triangles represent the locations of full network evaluations. Light green triangles represent partial observations where only the first node was evaluated by p-KGFN. Black, purple, and red squares correspond to the initial and final inferred best solutions identified by EIFN and p-KGFN, respectively. We use the different colors for each axis to represent different types of inputs and outputs of the network as follows: light blue denotes the original input $x$ to the network, dark blue denotes the output of the first node $y_1$, and dark navy denotes the output of the second node $y_2$.
• Figure 3: Function network structures in the numerical experiments: (a) Ackley and FreeSolv, (b) Manu-GP, and (c) Pharma.
• Figure 4: Optimization performance comparing between our proposed p-KGFN and benchmarks including EIFN, KGFN, TSFN, EI, KG and Random on four experiments: (a) Ackley, (b) Manu-GP, (c) FreeSolv, and (d) Pharma.
• Figure 5: Cost sensitivity analysis for Ackley problem with different costs (a) $c_1 = 1, c_2 = 1$; (b) $c_1=1, c_2=9$; and (c) $c_1 = 1, c_2 = 49$.

Theorems & Definitions (13)

• Theorem 1
• Theorem 1
• Lemma 1
• proof : Proof of Lemma \ref{['lemma:conth']}
• Lemma 2
• proof : Proof of Lemma \ref{['lemma:yK_bound']}
• Lemma 3
• proof : Proof of Lemma \ref{['lemma:dominated']}
• Lemma 4
• proof : Proof of Lemma \ref{['lemma:pointwise']}