Constrained Efficient Global Optimization of Expensive Black-box Functions

TL;DR

This work tackles constrained efficient global optimization for expensive black-box functions by introducing CONFIG, which leverages optimism in the face of uncertainty via lower confidence-bound surrogates for both the objective and the constraints. By solving an auxiliary constrained problem at each step, CONFIG attains cumulative regret bounds matching the unconstrained case and sublinear cumulative constraint-violation bounds, with kernels such as Matérn ($\nu> d/2$) and Squared Exponential yielding concrete convergence rates to the constrained optimum. The framework also provides a principled infeasibility-declaration mechanism when the problem is infeasible. Empirical results on GP-sampled, artificial, and building-control tasks demonstrate competitive performance against CEI and related baselines while offering stronger theoretical guarantees and the ability to declare infeasibility when appropriate.

Abstract

We study the problem of constrained efficient global optimization, where both the objective and constraints are expensive black-box functions that can be learned with Gaussian processes. We propose CONFIG (CONstrained efFIcient Global Optimization), a simple and effective algorithm to solve it. Under certain regularity assumptions, we show that our algorithm enjoys the same cumulative regret bound as that in the unconstrained case and similar cumulative constraint violation upper bounds. For commonly used Matern and Squared Exponential kernels, our bounds are sublinear and allow us to derive a convergence rate to the optimal solution of the original constrained problem. In addition, our method naturally provides a scheme to declare infeasibility when the original black-box optimization problem is infeasible. Numerical experiments on sampled instances from the Gaussian process, artificial numerical problems, and a black-box building controller tuning problem all demonstrate the competitive performance of our algorithm. Compared to the other state-of-the-art methods, our algorithm significantly improves the theoretical guarantees, while achieving competitive empirical performance.

Figures (5)

• Figure 1: A simple demonstration of different ways to construct the estimate of the feasible set, where we minimize an unknown function subject to that it is non-positive. Pessimistic estimate can guarantee feasibility with high probability. However, it can be easily seen that the pessimistic estimate is very tiny and can easily be empty if the initial several points are infeasible. With the certainty-equivalent estimate, the algorithm may easily get stuck due to lack of exploration. To the contrast, we construct an optimistic estimate of the feasible set using lower confidence bound functions, which contains the ground-truth feasible set with high probability.
• Figure 2: Cumulative regret and violation of different algorithms. The shaded area represents $\pm0.1\textsf{ standard deviation}$ and EPBO-$\rho$ represents EPBO with penalty $\rho$.
• Figure 3: The best suboptimality plus violation up to step $t$, that is $\min_{\tau\in[t]}[f(x_\tau)-f^*]^++[g(x_\tau)]^+$, of different algorithms. The shaded area represents $\pm0.1\textsf{ standard deviation}$. Here, EPBO-$\rho$ represents the EPBO algorithm with penalty term $\rho$. The SafeOPT has a higher initial value due to the restriction of starting from a feasible solution.
• Figure 4: Convergence of constrained regret for the collections of problems in Tab. \ref{['tab:arti_probs']}.
• Figure 5: Best normalized energy consumption plus normalized temperature deviation up to the current step. For SafeOPT, the initial required feasible solution is obtained based on domain knowledge.

Theorems & Definitions (19)

• Remark 1
• Remark 2
• Lemma 2.4: Theorem 2, chowdhury2017kernelized
• Definition 2.5
• Corollary 2.6
• Definition 2.7: Cumulative-Regret
• Definition 2.8: Cumulative-Violation
• Lemma 4.1
• Lemma 4.2: Lemma 4, chowdhury2017kernelized_arxiv
• Theorem 4.3