Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Local Constrained Bayesian Optimization

Jing Jingzhe, Fan Zheyi, Szu Hui Ng, Qingpei Hu

TL;DR

Theoretically, it is proved that LCBO achieves a convergence rate for the Karush-Kuhn-Tucker residual that depends polynomially on the dimension $d$ for common kernels under mild assumptions, offering a rigorous alternative to global BO where regret bounds typically scale exponentially.

Abstract

Bayesian optimization (BO) for high-dimensional constrained problems remains a significant challenge due to the curse of dimensionality. We propose Local Constrained Bayesian Optimization (LCBO), a novel framework tailored for such settings. Unlike trust-region methods that are prone to premature shrinking when confronting tight or complex constraints, LCBO leverages the differentiable landscape of constraint-penalized surrogates to alternate between rapid local descent and uncertainty-driven exploration. Theoretically, we prove that LCBO achieves a convergence rate for the Karush-Kuhn-Tucker (KKT) residual that depends polynomially on the dimension $d$ for common kernels under mild assumptions, offering a rigorous alternative to global BO where regret bounds typically scale exponentially. Extensive evaluations on high-dimensional benchmarks (up to 100D) demonstrate that LCBO consistently outperforms state-of-the-art baselines.

Local Constrained Bayesian Optimization

TL;DR

Theoretically, it is proved that LCBO achieves a convergence rate for the Karush-Kuhn-Tucker residual that depends polynomially on the dimension for common kernels under mild assumptions, offering a rigorous alternative to global BO where regret bounds typically scale exponentially.

Abstract

Bayesian optimization (BO) for high-dimensional constrained problems remains a significant challenge due to the curse of dimensionality. We propose Local Constrained Bayesian Optimization (LCBO), a novel framework tailored for such settings. Unlike trust-region methods that are prone to premature shrinking when confronting tight or complex constraints, LCBO leverages the differentiable landscape of constraint-penalized surrogates to alternate between rapid local descent and uncertainty-driven exploration. Theoretically, we prove that LCBO achieves a convergence rate for the Karush-Kuhn-Tucker (KKT) residual that depends polynomially on the dimension for common kernels under mild assumptions, offering a rigorous alternative to global BO where regret bounds typically scale exponentially. Extensive evaluations on high-dimensional benchmarks (up to 100D) demonstrate that LCBO consistently outperforms state-of-the-art baselines.
Paper Structure (34 sections, 17 theorems, 83 equations, 2 figures, 1 table, 1 algorithm)

This paper contains 34 sections, 17 theorems, 83 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Constrained Bayesian Optimization
  4. Local Bayesian Optimization
  5. Preliminaries
  6. Gaussian Process Fundamentals
  7. Modeling for Constrained Bayesian Optimization
  8. Notations and Definitions
  9. Proposed Algorithm
  10. Theoretical Analysis
  11. Assumptions
  12. Theoretical Analysis
  13. Experiments
  14. Baselines and Experimental Setup
  15. Baselines
  16. ...and 19 more sections

Key Result

Theorem 5.2

For any $\delta\in (0,1)$, suppose assumption:Gaussian-Process-Priorassumption:Compact-and-convex-Domainassumption:Regularity-Condition hold and the parameters in alg:lcbo are selected according to Then for the total iteration count $K > \tilde{K}_m$, there exists $\lambda_{k+ 1}$ such that, with probability at least $1-\delta$, the sequence $\{\bm{x}_k\}$ generated by alg:lcbo satisfies: Here,

Figures (2)

  • Figure 1: Optimization progress on synthetic tasks. The x-axis represents the number of function evaluations, while the y-axis shows the best feasible objective value found so far (lower is better). The solid curves represent the median performance over 10 independent runs. The shaded regions depict the interquartile range (25th–75th percentiles).
  • Figure 3: Best Feasible Objective vs. Iterations The x-axis represents the number of algorithm iterations, while the y-axis shows the best feasible objective value found so far (lower is better). We truncated the results to the minimum iteration count across the three batch schedules.

Theorems & Definitions (43)

  • Remark 1.1
  • Definition 3.1: $L$-smoothness
  • Definition 3.2: KKT Residuals
  • Definition 3.3: Error Function
  • Remark 4.1
  • Remark 5.1
  • Theorem 5.2
  • Corollary 1
  • Remark 5.3
  • Remark 6.1: Batch Size Schedule
  • ...and 33 more