Safety in safe Bayesian optimization and its ramifications for control

TL;DR

Safety in safe Bayesian optimization for controller tuning hinges on uncertainty bounds tied to the RKHS norm rather than practical, verifiable quantities. The paper introduces Lipschitz-only Safe BO (LoSBO), which relies on a known Lipschitz constant $L$ and bounded noise to certify safety without RKHS norm bounds, and a grid-free variant LoS-GP-UCB for continuous, moderately high-dimensional spaces. It also analyzes the limitations of SafeOpt-type methods due to theoretically grounded but impractical bounds and demonstrates that LoSBO maintains safety with favorable performance in experiments. The work advances safe online controller tuning and points to future directions such as handling multiple safety constraints and high-dimensional problems.

Abstract

A recurring and important task in control engineering is parameter tuning under constraints, which conceptually amounts to optimization of a blackbox function accessible only through noisy evaluations. For example, in control practice parameters of a pre-designed controller are often tuned online in feedback with a plant, and only safe parameter values should be tried, avoiding for example instability. Recently, machine learning methods have been deployed for this important problem, in particular, Bayesian optimization (BO). To handle safety constraints, algorithms from safe BO have been utilized, especially SafeOpt-type algorithms, which enjoy considerable popularity in learning-based control, robotics, and adjacent fields. However, we identify two significant obstacles to practical safety. First, SafeOpt-type algorithms rely on quantitative uncertainty bounds, and most implementations replace these by theoretically unsupported heuristics. Second, the theoretically valid uncertainty bounds crucially depend on a quantity - the reproducing kernel Hilbert space norm of the target function - that at present is impossible to reliably bound using established prior engineering knowledge. By careful numerical experiments we show that these issues can indeed cause safety violations. To overcome these problems, we propose Lipschitz-only Safe Bayesian Optimization (LoSBO), a safe BO algorithm that relies only on a known Lipschitz bound for its safety. Furthermore, we propose a variant (LoS-GP-UCB) that avoids gridding of the search space and is therefore applicable even for moderately high-dimensional problems.

Figures (4)

• Figure 1: Illustration of the required GP error bounds. Consider a fixed ground truth (solid black line), of which only finitely many samples are known (black dots). Applying GP regression leads to a posterior GP fully described by the posterior mean (solid blue line) and the posterior variance, from which a high-probability uncertainty set can be derived (shaded blue). Left: The ground truth is completely contained in the uncertainty set. Right: The ground truth violates the uncertainty bound around $x=1$. Figure from fiedler2024losbo.
• Figure 2: Illustration of LosBO being safe, while a safe set based on invalid uncertainty bounds leads to potential safety violations. The safe set of LoSBO (gray set) is determined by the constant $E$ (gray arrow) and the Lipschitz cone (orange). The GP mean and the confidence bounds are illustrated in blue. The points in the safe set given by the lower confidence bound are green if they are safe and red if they are unsafe. Figure from fiedler2024losbo.
• Figure 3: Comparison of LosBO and Real-$\beta$-SafeOpt in a well-specified and misspecified setting. Thick solid lines are the means over all functions and repetitions, thin solid lines are the means over all repetitions for each individual function, shaded area corresponds to one standard deviation over all runs. Figure from fiedler2024losbo.
• Figure 4: Illustration of one iteration of LoS-GP-UCB. Figure from fiedler2024losbo.