Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Constrained Bayesian optimization with merit functions

J. Wang, C. G. Petra, J. L. Peterson

TL;DR

A unified CBO algorithm that can be seen as extension to the popular expected constrained improvement (ECI) approach is proposed and demonstrated through numerical experiments on synthetic problems and a practical data-driven engineering design problem in the field of plasma physics.

Abstract

Bayesian optimization is a powerful optimization tool for problems where native first-order derivatives are unavailable. Recently, constrained Bayesian optimization (CBO) has been applied to many engineering applications where constraints are essential. However, several obstacles remain with current CBO algorithms that could prevent a wider adoption. We propose CBO algorithms using merit functions, such as the penalty merit function, in acquisition functions, inspired by nonlinear optimization methods, e.g., sequential quadratic programming. Merit functions measure the potential progress of both the objective and constraint functions, thus increasing algorithmic efficiency and allowing infeasible initial samples. The acquisition functions with merit functions are relaxed to have closed forms, making its implementation readily available wherever Bayesian optimization is. We further propose a unified CBO algorithm that can be seen as extension to the popular expected constrained improvement (ECI) approach. We demonstrate the effectiveness and efficiency of the proposed algorithms through numerical experiments on synthetic problems and a practical data-driven engineering design problem in the field of plasma physics.

Constrained Bayesian optimization with merit functions

TL;DR

A unified CBO algorithm that can be seen as extension to the popular expected constrained improvement (ECI) approach is proposed and demonstrated through numerical experiments on synthetic problems and a practical data-driven engineering design problem in the field of plasma physics.

Abstract

Bayesian optimization is a powerful optimization tool for problems where native first-order derivatives are unavailable. Recently, constrained Bayesian optimization (CBO) has been applied to many engineering applications where constraints are essential. However, several obstacles remain with current CBO algorithms that could prevent a wider adoption. We propose CBO algorithms using merit functions, such as the penalty merit function, in acquisition functions, inspired by nonlinear optimization methods, e.g., sequential quadratic programming. Merit functions measure the potential progress of both the objective and constraint functions, thus increasing algorithmic efficiency and allowing infeasible initial samples. The acquisition functions with merit functions are relaxed to have closed forms, making its implementation readily available wherever Bayesian optimization is. We further propose a unified CBO algorithm that can be seen as extension to the popular expected constrained improvement (ECI) approach. We demonstrate the effectiveness and efficiency of the proposed algorithms through numerical experiments on synthetic problems and a practical data-driven engineering design problem in the field of plasma physics.
Paper Structure (11 sections, 30 equations, 7 figures, 2 algorithms)

This paper contains 11 sections, 30 equations, 7 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Bayesian optimization
  3. Merit constrained Bayesian optimization
  4. Expected merit improvement
  5. A unified algorithm with with ECI
  6. Experiments
  7. Example with small feasible region
  8. Example with multiple constraints
  9. Four-dimensional problem
  10. ICF design problem
  11. Conclusion

Figures (7)

  • Figure 1: Contour plots for the objective function (left) and constraint function (right) for example 1. The feasible region is marked on the constraint plot, along with the global optimum ($*$ sign).
  • Figure 2: Optimization result of MCBO, UCBO, ECI (median) and ADMMBO (median) algorithms for example 1, shown in figure \ref{['fig:ex1contour']}. The best feasible objective is plotted against the number of iterations, or equivalently, number of samples.
  • Figure 3: Contour plots for the objective function (left) and the two constraint functions (middle and right) for example 2. The feasible region is marked with black line on the objective contour, along with the global optimum ($*$ sign). The contour curve $c(x)=0$ is in black on each constraint contour plot as well.
  • Figure 4: Optimization result of MCBO, UCBO, ECI (median), and ADMMBO (median) algorithms for example 2, shown in figure \ref{['fig:ex2contour']}. The best feasible objective is plotted against the number of iterations.
  • Figure 5: Optimization result of MCBO, UCBO, ECI (median) and ADMMBO (median) algorithms for example 3. The best feasible objective is plotted against the number of iterations.
  • ...and 2 more figures