Task-free Adaptive Meta Black-box Optimization

TL;DR

ABOM removes the reliance on handcrafted training task distributions by embedding online, task-specific parameter adaptation inside the evolutionary optimization loop. It replaces discrete meta-optimizer components with a differentiable, attention-based parametrization of selection, crossover, and mutation, updated via gradient descent from the elite archive of generated solutions. Theoretical guarantees include exploration and global convergence under reasonable assumptions, while empirical results on synthetic BBOB and realistic UAV benchmarks demonstrate competitive, zero-shot performance without prior task distributions. The approach also yields interpretable search dynamics through learned attention matrices and is GPU-friendly, offering a practical, task-free path for adaptive meta-black-box optimization.

Abstract

Handcrafted optimizers become prohibitively inefficient for complex black-box optimization (BBO) tasks. MetaBBO addresses this challenge by meta-learning to automatically configure optimizers for low-level BBO tasks, thereby eliminating heuristic dependencies. However, existing methods typically require extensive handcrafted training tasks to learn meta-strategies that generalize to target tasks, which poses a critical limitation for realistic applications with unknown task distributions. To overcome the issue, we propose the Adaptive meta Black-box Optimization Model (ABOM), which performs online parameter adaptation using solely optimization data from the target task, obviating the need for predefined task distributions. Unlike conventional metaBBO frameworks that decouple meta-training and optimization phases, ABOM introduces a closed-loop adaptive parameter learning mechanism, where parameterized evolutionary operators continuously self-update by leveraging generated populations during optimization. This paradigm shift enables zero-shot optimization: ABOM achieves competitive performance on synthetic BBO benchmarks and realistic unmanned aerial vehicle path planning problems without any handcrafted training tasks. Visualization studies reveal that parameterized evolutionary operators exhibit statistically significant search patterns, including natural selection and genetic recombination.

Figures (16)

• Figure 1: Conceptual comparison: (Left) MetaBBO methods learn meta-strategies from task distributions but depend on handcrafted training tasks; (Right) Our framework performs adaptive parameter learning using self-generated optimization data, eliminating task distribution dependency.
• Figure 2: Workflow of ABOM: (Top) Adaptive optimization loop: Initialization, reproduction, evaluation, elitism, and parameter adaptation; (Bottom) Meta-strategies for reproduction: Attention-based evolutionary operators, including selection, crossover, and mutation.
• Figure 3: Performance on 28 UAV problems: (Left) Convergence curve of average normalized cost across all problems. Costs (lower is better) are min-max normalized for each case. Detailed results are shown in the Appendix \ref{['er']}; (Right) Average runtime (GPU seconds) over 30 independent runs.
• Figure 4: Learned selection and mutation matrices of ABOM on BBOB functions $f_4$, $f_{11}$, and $f_{24}$ ($d=30$) at Generation 1, 500, and 1000. For the selection matrix, axes represent individuals ranked by their fitness values (0 is the best). For the mutation matrix, axes represent gene (variable) indices.
• Figure 5: Sensitivity analysis of key hyperparameters on the BBOB suite with $d=30$: Algorithm performance across different settings for population size ($N$), hidden dimension ($d_M$), crossover dropout rate ($p_C$), and mutation dropout rate ($p_M$). The learning rate analysis is in Appendix \ref{['sectionsalr']}.

Theorems & Definitions (9)

• Corollary 1: Exploration Guarantee
• Corollary 2: Progress Guarantee
• Theorem 3.1: Global Convergence
• Lemma 1
• Proof 1
• Lemma 2
• Proof 2
• Theorem D.1
• Proof 3