Greedy Restart Schedules: A Baseline for Dynamic Algorithm Selection on Numerical Black-box Optimization Problems

TL;DR

This work tackles the challenge of combining multiple solvers for numerical black-box optimization by proposing Greedy Restart Schedule (GRS), a data-driven, model-free static restart strategy. GRS selects the next algorithm by maximizing the expected solved-proportion per runtime across the current distribution of unsolved problems, updating this distribution after each restart, and ultimately producing a fixed schedule. On the BBOB testbed, GRS substantially closes the gap between the single best solver and the virtual best solver, performing close to an oracle across evaluation protocols and showing favorable comparisons to state-of-the-art hybrid heuristics, especially at lower dimensions and budgets. The results indicate that a simple, data-derived restart strategy can serve as a strong baseline for dynamic algorithm selection and can be extended with richer portfolios or landscape-feature-informed predictions for further gains.

Abstract

In many optimization domains, there are multiple different solvers that contribute to the overall state-of-the-art, each performing better on some, and worse on other types of problem instances. Meta-algorithmic approaches, such as instance-based algorithm selection, configuration and scheduling, aim to close this gap by extracting the most performance possible from a set of (configurable) optimizers. In this context, the best performing individual algorithms are often hand-crafted hybrid heuristics which perform many restarts of fast local optimization approaches. However, data-driven techniques to create optimized restart schedules have not yet been extensively studied. Here, we present a simple scheduling approach that iteratively selects the algorithm performing best on the distribution of unsolved training problems at time of selection, resulting in a problem-independent solver schedule. We demonstrate our approach using well-known optimizers from numerical black-box optimization on the BBOB testbed, bridging much of the gap between single and virtual best solver from the original portfolio across various evaluation protocols. Our greedy restart schedule presents a powerful baseline for more complex dynamic algorithm selection models.

Figures (7)

• Figure 1: Runtime profiles depicting the proportion of problems solved for A1, A2, and a schedule alternating A1 and A2, for problems P1, P2, and the uniform problem distribution (All). While the simple schedule in this example cannot beat any of its component algorithms on the individual problems, it can close a significant gap to the best respective solver. On the overall problem distribution, it outperforms A1 and A2 (which perform identically).
• Figure 2: Runtime profiles for $d \in \{2,3,5,10\}$ for all algorithms, the synthetic oracle algorithm (VBS), and the greedy restart schedule (GRS). The schedule beats all individual algorithms overall and approaches the performance of the oracle.
• Figure 3: Runtime heatmaps depicting the relERT for all problems with $d=10$. Our GRS provides the best balance of performance across all problems.
• Figure 4: GRS composition for $d \in \{2,3,5,10\}$ showing the first $1\,000$ restarts in each scenario. The algorithms are stacked in order of execution, showing the expected evaluations w.r.t. the BBOB problem set.
• Figure 5: Runtime profiles for GRS, the synthetic oracle algorithm (VBS), and the hybrid optimizers HCMA and BIPOP-CMA-ES (a) across all functions for $d \in \{2,3,5\}$ and (b) per function group and for all functions in $d=10$. GRS beats the other approaches for lower dimensions, and often has a performance advantage on low budgets compared to the other off-the-shelf solvers.