Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints

Yunsheng Tian, Ane Zuniga, Xinwei Zhang, Johannes P. Dürholt, Payel Das, Jie Chen, Wojciech Matusik, Mina Konaković Luković

TL;DR

This work tackles Bayesian optimization under unknown physical constraints by revealing that optimal designs often reside on the boundary between feasible and infeasible regions. It introduces BE-CBO, which couples a Gaussian process surrogate for the objective with Deep Ensembles for binary constraint modeling and a dynamic boundary exploration strategy. By training the constraint model with variational inference and enforcing a boundary-aware constrained acquisition, BE-CBO achieves superior performance across synthetic and real-world benchmarks and provides robust uncertainty quantification. The approach enables efficient exploration of the feasible boundary, accelerating discovery of high-performing designs under unknown constraints, with practical implications for engineering and materials science experimentation.

Abstract

Bayesian optimization has been successfully applied to optimize black-box functions where the number of evaluations is severely limited. However, in many real-world applications, it is hard or impossible to know in advance which designs are feasible due to some physical or system limitations. These issues lead to an even more challenging problem of optimizing an unknown function with unknown constraints. In this paper, we observe that in such scenarios optimal solution typically lies on the boundary between feasible and infeasible regions of the design space, making it considerably more difficult than that with interior optima. Inspired by this observation, we propose BE-CBO, a new Bayesian optimization method that efficiently explores the boundary between feasible and infeasible designs. To identify the boundary, we learn the constraints with an ensemble of neural networks that outperform the standard Gaussian Processes for capturing complex boundaries. Our method demonstrates superior performance against state-of-the-art methods through comprehensive experiments on synthetic and real-world benchmarks. Code available at: https://github.com/yunshengtian/BE-CBO

Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints

TL;DR

This work tackles Bayesian optimization under unknown physical constraints by revealing that optimal designs often reside on the boundary between feasible and infeasible regions. It introduces BE-CBO, which couples a Gaussian process surrogate for the objective with Deep Ensembles for binary constraint modeling and a dynamic boundary exploration strategy. By training the constraint model with variational inference and enforcing a boundary-aware constrained acquisition, BE-CBO achieves superior performance across synthetic and real-world benchmarks and provides robust uncertainty quantification. The approach enables efficient exploration of the feasible boundary, accelerating discovery of high-performing designs under unknown constraints, with practical implications for engineering and materials science experimentation.

Abstract

Bayesian optimization has been successfully applied to optimize black-box functions where the number of evaluations is severely limited. However, in many real-world applications, it is hard or impossible to know in advance which designs are feasible due to some physical or system limitations. These issues lead to an even more challenging problem of optimizing an unknown function with unknown constraints. In this paper, we observe that in such scenarios optimal solution typically lies on the boundary between feasible and infeasible regions of the design space, making it considerably more difficult than that with interior optima. Inspired by this observation, we propose BE-CBO, a new Bayesian optimization method that efficiently explores the boundary between feasible and infeasible designs. To identify the boundary, we learn the constraints with an ensemble of neural networks that outperform the standard Gaussian Processes for capturing complex boundaries. Our method demonstrates superior performance against state-of-the-art methods through comprehensive experiments on synthetic and real-world benchmarks. Code available at: https://github.com/yunshengtian/BE-CBO
Paper Structure (62 sections, 10 equations, 21 figures, 1 table)

This paper contains 62 sections, 10 equations, 21 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Constrained Bayesian Optimization
  4. Safe Bayesian Optimization
  5. Neural Networks for Classification
  6. Preliminaries
  7. Proposed Method
  8. Modeling Constraints
  9. Deep Ensembles
  10. Benefits of Deep Ensembles
  11. Training Deep Ensembles
  12. Boundary Exploration
  13. Experimental Evaluation
  14. Algorithms
  15. Benchmark Problems
  16. ...and 47 more sections

Figures (21)

  • Figure 1: Comparison of modeling constraints between Gaussian Processes (GP) and Deep Ensembles (DE) on the LSQ problem with three sets of random sample evaluations. Green dots represent feasible designs and red dots represent infeasible designs. We observe that DE tends to be more robust than GP in capturing complex boundaries. (a) Failure case of GP; (b) Successful case of GP; (c) Both GP and DE are not fitted well due to the poor sample distribution but DE is closer to the ground truth.
  • Figure 2: Example evaluation of the LSQ synthetic benchmark problem guided by our algorithm BE-CBO. Top row illustrates the objective function, while the bottom row illustrates the classifier. In the left plot, the ground truth is shown with the boundary and optimal solution. In the second plot, the state of the surrogate is shown for both the constraint and the objective, based on the first initial 10 samples. The following three plots demonstrate the state of the surrogate models for the objective and the constraint after 50, 120, and 200 evaluations.
  • Figure 3: Quantitative comparison of different algorithms including BE-CBO on all benchmark problems. The current best value is shown w.r.t. the number of function evaluations. Each experiment has 10 initial random samples and 200 total evaluations. The curve is averaged over 10 different random seeds and the standard deviation is shown as a shaded region.
  • Figure 4: Qualitative comparison of sample distributions from different algorithms on the Simionescu benchmark. Left: The true function landscape, where darker color means a higher objective value and the white region is infeasible. Right: The predicted function landscape (top: CEI, middle: SCBO, bottom: BE-CBO) where darker color means a higher objective value and the contour represents the feasibility boundary (feasible inside, infeasible outside). Grey: initial samples, black: evaluated samples guided by each algorithm, and red: the global optima.
  • Figure 5: Feasibility ratio comparison of different algorithms including our BE-CBO on synthetic test functions and real-world problems. The error bar charts represent the mean and variance over 10 different random seeds.
  • ...and 16 more figures