Branching Adaptive Surrogate Search Optimization (BASSO)

TL;DR

The paper tackles black-box global optimization in high dimensions by introducing BASSO, a framework that combines sequential partitioning of the search domain with surrogate-based sampling within subregions. It provides finite-time analyses showing that, under two key assumptions, the expected number of function evaluations can scale linearly with dimension, offering a principled route toward scalability. Empirically, it explores numerous variants and demonstrates that a regularized quadratic surrogate together with a one-dimensional Gaussian process for adaptive subregion probabilities often yields the best performance, while surrogate-assisted approaches generally outperform uniform sampling. The work offers insights into the exploration-exploitation trade-offs needed to push high-dimensional surrogate-based optimization closer to theoretical scalability, and it benchmarks against established solvers to illustrate practical competitiveness and limitations.

Abstract

Global optimization of black-box functions is challenging in high dimensions. We introduce a conceptual adaptive random search framework, Branching Adaptive Surrogate Search Optimization (BASSO), that combines partitioning and surrogate modeling for subregion sampling. We present a finite-time analysis of BASSO, and establish conditions under which it is theoretically possible to scale to high dimensions. While we do not expect that any implementation will achieve the theoretical ideal, we experiment with several BASSO variations and discuss implications on narrowing the gap between theory and implementation. Numerical results on test problems are presented.

Figures (12)

• Figure 1: Sampling from a Boltzmann distribution parameterized by temperature induces the densities to focus more on the optimum as the temperature decreases. In BASSO, the adaptive subregion probability parameter $\tilde{p}$ impacts the sampling distribution.
• Figure 2: Empirical density range distribution (gray histogram) and empirical cumulative range distribution (dashed red line) for 100 points sampled independently, where the domain is partitioned into three subregions (i.e., intervals). In (a), the adaptive subregion probability is $\tilde{p}=[0.33,0.33,0.33]$, representing a uniform distribution. In (b), the adaptive subregion probability $\tilde{p}=[0.1,0.8,0.1]$. Once a subregion is selected, a point is sampled uniformly on that interval.
• Figure 3: Performance profiles for 15 algorithms on 12 benchmark problems in dimensions 20 and 50, with $\tau = 0.001$ in (a) and $\tau = 0.00001$ in (b).
• Figure 4: Performance profiles for 11 algorithms on 18 benchmark problems in dimensions 20, 50 and 100, with $\tau = 0.001$ in (a) and $\tau = 0.00001$ in (b).
• Figure 5: BASSO variations (Aa, Ba, Ca) with adaptive subregion probabilities in \ref{['eq:ptildeincumbent']}, observed best function value (a). When the left-hand side (LHS) of the probability ratio in \ref{['ratioassumption']} with $t_z$ as the incumbent function value is larger than the right-hand side (RHS) with $t_{z'}$ as the 20% quantile of the observed objective function values, $t_z < t_{z'}$, Assumption 1.3 is satisfied.

Theorems & Definitions (4)

• Theorem 1
• Theorem 2
• Corollary 1
• Corollary 2