Towards safe Bayesian optimization with Wiener kernel regression

TL;DR

The paper addresses safe Bayesian optimization under Gaussian measurement noise by deriving a Wiener kernel regression (WK)–based probabilistic error bound that separates noise-induced uncertainty from the surrogate’s posterior variance. The main result is a bound $_{ ext{WK}}(x)= B \sqrt{\sigma_{ ext{GP}}^2(x)-\sigma_{ ext{WK}}^2(x)} + \beta_{ ext{WK}}(\delta)(x)$ with $\beta_{ ext{WK}}(\delta)=\sqrt{2\ln(2/\delta)}$, proven to be tighter than existing bounds and to imply enlarged safe regions. The authors show that $\sigma_{ ext{WK}}^2(x) \le \sigma_{ ext{GP}}^2(x)$, extend the bound to sub-Gaussian noise, and provide a qualitative demonstration via a numerical safe BO example where WK yields faster safe-region growth and lower regret. Overall, the work offers tighter safety guarantees for BO and highlights the practical impact of explicitly accounting for noise structure through WK regression in safety-critical optimization settings.

Abstract

Bayesian Optimization (BO) is a data-driven strategy for minimizing/maximizing black-box functions based on probabilistic surrogate models. In the presence of safety constraints, the performance of BO crucially relies on tight probabilistic error bounds related to the uncertainty surrounding the surrogate model. For the case of Gaussian Process surrogates and Gaussian measurement noise, we present a novel error bound based on the recently proposed Wiener kernel regression. We prove that under rather mild assumptions, the proposed error bound is tighter than bounds previously documented in the literature, leading to enlarged safety regions. We draw upon a numerical example to demonstrate the efficacy of the proposed error bound in safe BO.

Figures (3)

• Figure 1: Evolution of the cumulative regret and the size of the safe region $\glsd{xsafe}$ over learning steps for all comparison methods. Shaded areas show the $75\%$-confidence intervals over all runs. Thick lines represent the mean.
• Figure 2: Evolution of the bound parameters $\beta_{\mathrm{WK}}(\delta)$ (see Theorem \ref{['th:bound']}) and $\beta_1(\delta)$, $\beta_2(\delta)$ (see Table \ref{['tab:bounds']}) over learning steps $t$ for $\delta = 0.001$. The shaded area shows the maximum and minimum of $\beta_1(\delta)$ over all runs.
• Figure 3: Comparison of the learning progress based on the proposed error bound $_{\mathrm{WK}}$ from Theorem \ref{['th:bound']} (upper row) and based on the error bound $_{1}$ from Table \ref{['tab:bounds']}abbasi2013online (lower row) for learning steps $t=8$ (left column), $t=14$ (middle column), and $t=20$ (right column).

Theorems & Definitions (21)

• Definition 1: Kernel function
• Definition 2: Aronszajn1950Berlinet2004
• Lemma 1: Error bound with noise-free data Fasshauer2011
• Definition 3: Probabilistic error bound
• Lemma 2: Safety constraint satisfaction
• proof
• Remark 1: Overview of Existing Error Bounds
• Lemma 3: Extension of Lemma \ref{['lem:bound_noise_free']}
• proof
• Theorem 1: Wiener kernel error bound