Every Call is Precious: Global Optimization of Black-Box Functions with Unknown Lipschitz Constants

TL;DR

This work tackles global maximization of expensive black-box functions with unknown Lipschitz constants. It introduces Every Call is Precious (ECP), an adaptive algorithm that avoids estimating $k$ by expanding an acceptance region via a growing sequence of $\\varepsilon_t$, ensuring evaluations stay in regions likely to contain maximizers. Theoretical guarantees include no-regret in the infinite-budget setting and minimax-optimal regret in finite budgets, alongside finite-time computational guarantees. Extensive experiments on 30 non-convex problems show ECP outperforms 10 benchmark methods across disciplines and budgets, with robust performance and publicly available code, making it a competitive approach for practical global optimization under unknown smoothness.

Abstract

Optimizing expensive, non-convex, black-box Lipschitz continuous functions presents significant challenges, particularly when the Lipschitz constant of the underlying function is unknown. Such problems often demand numerous function evaluations to approximate the global optimum, which can be prohibitive in terms of time, energy, or resources. In this work, we introduce Every Call is Precious (ECP), a novel global optimization algorithm that minimizes unpromising evaluations by strategically focusing on potentially optimal regions. Unlike previous approaches, ECP eliminates the need to estimate the Lipschitz constant, thereby avoiding additional function evaluations. ECP guarantees no-regret performance for infinite evaluation budgets and achieves minimax-optimal regret bounds within finite budgets. Extensive ablation studies validate the algorithm's robustness, while empirical evaluations show that ECP outperforms 10 benchmark algorithms including Lipschitz, Bayesian, bandits, and evolutionary methods across 30 multi-dimensional non-convex synthetic and real-world optimization problems, which positions ECP as a competitive approach for global optimization.

Figures (7)

• Figure 1: Selected figures of various considered non-convex objective functions with two dimensions.
• Figure 2: An illustration of the acceptance region (orange), which is determined based on the 8 evaluated points on a non-convex, single-dimensional objective function (black). The $\varepsilon_8$ controls the slopes of the blue functions, directly impacting the acceptance region.
• Figure 3: Violin plots showing the ranking distributions of diverse optimization algorithms, ordered by increasing median ranking, across 30 non-convex, multi-dimensional synthetic and real-world problems. The results are based on a budget of $n=50$ for the different algorithms, with maxima averaged over 100 repetitions.
• Figure 4: Figures of the various considered non-convex 2-dimensional objective functions.
• Figure 5: Ablation Study on the Constant $\varepsilon_1 > 0$ of ECP with fixed $C=10^{3}$ and $\tau=10^{-3}$ on various real-world and synthetic non-convex multi-dimensional optimization problems.

Theorems & Definitions (24)

• Definition 1
• Proposition 1
• Definition 2
• Remark 1
• Definition 3
• Definition 4
• Lemma 1
• Proposition 2
• Proposition 3
• Proposition 4