CMA-ES for Safe Optimization

TL;DR

The paper tackles safe optimization where unsafe evaluations pose risk by proposing Safe CMA-ES, an extension of CMA-ES that uses a Gaussian-process–based estimate of Lipschitz constants for safety functions on a transformed space. It constructs a safe region from evaluated data and Lipschitz estimates, projecting generated samples to the nearest safe point to avoid unsafe evaluations, while initializing distributions with safe seeds and adapting the search via the transform $\phi$. Empirical results on benchmark functions show that Safe CMA-ES suppresses unsafe evaluations and maintains competitive optimization performance, outperforming grid-based SafeOpt variants in higher dimensions and avoiding unsafe evaluations more effectively than violation-avoidance baselines. The work highlights the practical benefits of combining Lipschitz-based safety with CMA-ES’ robustness, though it relies on Lipschitz continuity and synthetic experiments, suggesting future work for discontinuous safety and real-world testing.

Abstract

In several real-world applications in medical and control engineering, there are unsafe solutions whose evaluations involve inherent risk. This optimization setting is known as safe optimization and formulated as a specialized type of constrained optimization problem with constraints for safety functions. Safe optimization requires performing efficient optimization without evaluating unsafe solutions. A few studies have proposed the optimization methods for safe optimization based on Bayesian optimization and the evolutionary algorithm. However, Bayesian optimization-based methods often struggle to achieve superior solutions, and the evolutionary algorithm-based method fails to effectively reduce unsafe evaluations. This study focuses on CMA-ES as an efficient evolutionary algorithm and proposes an optimization method termed safe CMA-ES. The safe CMA-ES is designed to achieve both safety and efficiency in safe optimization. The safe CMA-ES estimates the Lipschitz constants of safety functions transformed with the distribution parameters using the maximum norm of the gradient in Gaussian process regression. Subsequently, the safe CMA-ES projects the samples to the nearest point in the safe region constructed with the estimated Lipschitz constants. The numerical simulation using the benchmark functions shows that the safe CMA-ES successfully performs optimization, suppressing the unsafe evaluations, while the existing methods struggle to significantly reduce the unsafe evaluations.

Figures (8)

• Figure 1: The safe region on the safety function $s(\boldsymbol{x}) = x_1^2 + 10 x_2^2$ with the safety threshold $h=5$ with four safe seeds. The distribution parameters are given by $\boldsymbol{m} = \mathbf{0}$, $\sigma=1$, and $\boldsymbol{C}= \mathrm{diag}((1, 0.1)^\mathrm{T})$. We generated safe seeds uniformly at random in the range $[-1,1]^2$ plotted as a white dotted box. The white circles and the orange line show the safe region and the border of the safety constraint. The left figure shows the safe region on the composition function $s_j \circ \phi^{-1}$, and the center figure shows the safe region on the safety function $s_j$. The right figure shows the safe region on the safety function $s_j$ computed with the Lipschitz constant of the safety function $s_j$ instead of the composition function $s_j \circ \phi^{-1}$.
• Figure 2: Transitions of the best evaluation value and the number of evaluations of unsafe solutions with safety function $s(\boldsymbol{x}) = f(\boldsymbol{x})$. We plot the medians and interquartile ranges over 50 trials.
• Figure 3: Transitions of the best evaluation value and the number of evaluations of unsafe solutions with safety function $s(\boldsymbol{x}) = x_1$. We plot the medians and interquartile ranges over 50 trials.
• Figure 4: The computational time for performing five updates in each method. We plot the average time over three trials.
• Figure 5: The performance comparison of the safe CMA-ES with the swarm-based SafeOpt and the CMA-ES with violation avoidance. We plot the medians and interquartile ranges of the best evaluation value and the number of unsafe evaluations .