Local Bayesian Optimization for Controller Tuning with Crash Constraints

TL;DR

This work extends a recently proposed local variant of BO to include crash constraints, where the controller can only be successfully evaluated in an a-priori unknown feasible region, and demonstrates the efficiency of the proposed method through simulations and hardware experiments.

Abstract

Controller tuning is crucial for closed-loop performance but often involves manual adjustments. Although Bayesian optimization (BO) has been established as a data-efficient method for automated tuning, applying it to large and high-dimensional search spaces remains challenging. We extend a recently proposed local variant of BO to include crash constraints, where the controller can only be successfully evaluated in an a-priori unknown feasible region. We demonstrate the efficiency of the proposed method through simulations and hardware experiments. Our findings showcase the potential of local BO to enhance controller performance and reduce the time and resources necessary for tuning.

Figures (8)

• Figure 1: The controller tuning process with bo. The objective $f$ is evaluated in closed-loop. The controller $\pi_{\bm{x}}$ has tuning parameters $\bm{x} \in \mathcal{X}$ and bo searches for the optimal parameterization. No function value is available if an experiment crashes, $\bm{x} \not\in \mathcal{X}_S \subseteq \mathcal{X}$.
• Figure 2: Left: A Gaussian process posterior (top) and its derivative (bottom). Right: The posterior with an additional virtual observation in the crash region $\mathcal{X}_C$. The crashed evaluation (red cross) cannot be evaluated, and a virtual observation is added instead. In this example, the virtual data point modifies the posterior such that the minimum of the posterior is not inside the infeasible region $\mathcal{X}_C$, and the gradient points away from it.
• Figure 3: Diagram of the coupled tank system, with controllable pump and valves.
• Figure 4: Simulation results on crash constrained controller tuning problems:crashgibo is able to solve $2$-(PI),4-(cascaded PI),6-(mpc) and 8-dimensional (lqi) controller tuning problems in a handful of evaluations. The controller performance shown as the median over $10$ runs with randomized initial controller parameters. The individual runs are shown as thin lines and demonstrate the low variability in tuning results with the proposed method. As baseline (dashed line), we draw parameters uniformly at random from the search domain and chose the best evaluation. The number of evaluation is the same as for crashgibo. Please note that the objective functions are different between the PI and the MIMO (lqi and mpc) controllers.
• Figure 5: Evaluations in the parameter space for PI ($V_2 < 8 \, \mathrm{l}$) (left) and PI ($V_2 < 7 \, \mathrm{l}$) (right). We show the first eight improvement steps and the corresponding evaluation locations $X$, improvement steps $X_*$ and crashes $\hat{X}$. Due to the tighter constraints on the right, the feasible optima changes. The virtual observations change the gradient such that the algorithm can estimate this new local optimum. The majority of the parameter space remains unexplored, increasing data-efficiency.