ECPv2: Fast, Efficient, and Scalable Global Optimization of Lipschitz Functions

TL;DR

ECPv2 is proposed, a scalable and theoretically grounded algorithm for global optimization of Lipschitz-continuous functions with unknown Lipschitz constants that retains ECP's no-regret guarantees with optimal finite-time bounds and expands the acceptance region with high probability.

Abstract

We propose ECPv2, a scalable and theoretically grounded algorithm for global optimization of Lipschitz-continuous functions with unknown Lipschitz constants. Building on the Every Call is Precious (ECP) framework, which ensures that each accepted function evaluation is potentially informative, ECPv2 addresses key limitations of ECP, including high computational cost and overly conservative early behavior. ECPv2 introduces three innovations: (i) an adaptive lower bound to avoid vacuous acceptance regions, (ii) a Worst-m memory mechanism that restricts comparisons to a fixed-size subset of past evaluations, and (iii) a fixed random projection to accelerate distance computations in high dimensions. We theoretically show that ECPv2 retains ECP's no-regret guarantees with optimal finite-time bounds and expands the acceptance region with high probability. We further empirically validate these findings through extensive experiments and ablation studies. Using principled hyperparameter settings, we evaluate ECPv2 across a wide range of high-dimensional, non-convex optimization problems. Across benchmarks, ECPv2 consistently matches or outperforms state-of-the-art optimizers, while significantly reducing wall-clock time.

Figures (15)

• Figure 1: Comparison of Lipschitz optimization methods on Rosenbrock with $d \in \{3, 100, 200, 300, 500\}$. Each star shows the mean performance across dimensions after $n=200$ evaluations, averaged over 100 runs. ECPv2 uses hyperparameters ($\beta = 5$, $\delta = 2/3$, $m = 8$).
• Figure 2: Ablation study on the projection dimension $m$ in ECPv2, using fixed parameters $\delta = 2/3$ and $\beta = 5$. For each benchmark function, every method is allocated $n = 1000$ evaluations and performance is averaged over 100 independent runs. Each point reports the mean final best value with $\pm$ half a standard deviation.
• Figure 3: JL scaling term $f(\delta) = \frac{1}{\delta^2 - \delta^3}$ as a function of distortion $\delta$. The function attains a minimum at $\delta = \frac{2}{3}$, yielding the most distortion-efficient embedding.
• Figure 4: Projection dimension $d'$ as a function of the number of evaluations $n$ for various values of $\delta \in \{0.1, 0.2, 0.3, 0.4, 0.5, 0.6 \}$, with $\beta = 5$ fixed.
• Figure 5: Histogram of pairwise distance distortions aggregated over 1000 random trials. Each histogram corresponds to a different value of $\beta$, with $n = 100$, $d = 1000$, and distortion bound $\delta = 0.5$. A thick blue horizontal line marks the acceptable distortion interval $[1 - \delta,\ 1 + \delta]$. As $\beta$ increases (and with it the projection dimension $d'$), the distortion distribution becomes more concentrated around 1, consistent with Lemma \ref{['lemma:distoriton_bound']}.

Theorems & Definitions (31)

• Definition 1: No Regret
• Proposition 1: Minimax Regret Lower Bound bull2011convergence
• Definition 2
• Definition 3
• Lemma 1: Lower Bound on $\varepsilon_t$
• Lemma 2
• Lemma 3
• Lemma 4
• Lemma 5
• Corollary 1