Bayesian Optimization with Lower Confidence Bounds for Minimization Problems with Known Outer Structure

TL;DR

This work extends Bayesian optimization to grey-box problems where the outer loss is known while the inner model is unknown and learned from noisy observations. It introduces a structure-exploiting lower confidence bound (LCB) that relies on tractable model confidence sets for the unknown inner function and defines a corresponding acquisition function that lower-bounds the true objective with high probability. The authors provide a regret analysis for the case of linear-in-parameters models with convex losses, showing a bound that scales with dimension and sample size similar to linear bandits, and they illustrate clear performance gains over structure-agnostic approaches in a linear-model example and a high-dimensional iterative learning control scenario. The framework unifies standard LCB behavior in linear and GP settings while enabling principled exploration and exploitation that leverages known outer structure, with practical implications for control, robotics, and grey-box optimization.

Abstract

This paper considers Bayesian optimization (BO) for problems with known outer problem structure. In contrast to the classic BO setting, where the objective function itself is unknown and needs to be iteratively estimated from noisy observations, we analyze the case where the objective has a known outer structure - given in terms of a loss function - while the inner structure - an unknown input-output model - is again iteratively estimated from noisy observations of the model outputs. We introduce a novel lower confidence bound algorithm for this particular problem class which exploits the known outer problem structure. The proposed method is analyzed in terms of regret for the special case of convex loss functions and probabilistic parametric models which are linear in the uncertain parameters. Numerical examples illustrate the superior performance of structure-exploiting methods compared to structure-agnostic approaches.

Figures (5)

• Figure 1: Samples from the probabilistic model for the unknown objective in the classic, structure-agnostic BO setting (left) and the structure-exploiting method (right) for Example \ref{['ex:minimal']}. The top row shows the prior model, while the bottom row shows the model after evaluating at $u_1 = 0$ and observing the model output $z_1 = (0.4, 0.1)$ for the proposed method and the objective value $\varphi_1 = 0.161$ for the classic BO method.
• Figure 2: Samples from the unknown objective in the classic, structure-agnostic BO setting (left) and the structure-exploiting method (right) as well as the corresponding lower confidence bounds for different confidence levels for Example \ref{['ex:minimal']}. The top row shows the prior model, while the middle row shows the model after evaluating at $u_1 = -1$ and observing the model output $y_1 = (1.5, 1)$ for the proposed method and the objective value $\varphi_1 = 2.35$ for the classic BO method. The bottom row shows the model after the additional evaluation at $u_2 = 1$ and observing $y_2 = (-0.7, 0.1)$ and $\varphi_2 = 0.491$, respectively.
• Figure 3: Intermediate iterates (grey), as well as the final iterate (orange) and the optimal solution (blue). For the Thompson sampling variants, the iterates of one of the independent repetitions are shown.
• Figure 4: Instantaneous regret for the deterministic simulation. For the two Thompson sampling variants the experiment is repeated for 100 independent runs. The right plot shows a zoom to illustrate that the zero-order ILC method converges to a nonzero regret.
• Figure 5: Cumulative regret for the deterministic simulation. For the two Thompson sampling variants, the median as well as the minimum and maximum cumulative regret is shown.

Theorems & Definitions (17)

• Definition 3: Acquisition function
• Lemma 4: Lower confidence bound
• Proposition 5
• Lemma 6: Boundedness
• Lemma 8: Linear loss
• Lemma 9: Convex subproblems
• Remark 10: Gaussian processes
• Example 1
• Lemma 11
• Theorem 15: Simultaneous model confidence bound for LIP models