Neural Exploratory Landscape Analysis for Meta-Black-Box-Optimization

TL;DR

This work tackles the reliance of MetaBBO on hand-crafted landscape features by introducing NeurELA, a neural Exploratory Landscape Analysis framework that dynamically profiles optimization status via a two-stage attention network $\Lambda_{\theta}$. Trained with multitask neuroevolution over a multi-task space $\Omega$, NeurELA enables zero-shot generalization to unseen MetaBBO tasks and can be fine-tuned to improve task-specific performance. Experiments show NeurELA outperforms traditional ELA, maintains robustness across noisy and realistic problems, and offers competitive efficiency with scalable attention mechanisms. Overall, NeurELA moves MetaBBO toward fully autonomous optimization with reduced expert tuning, albeit with acknowledged limitations in training efficiency and non-differentiable objectives.

Abstract

Recent research in Meta-Black-Box Optimization (MetaBBO) have shown that meta-trained neural networks can effectively guide the design of black-box optimizers, significantly reducing the need for expert tuning and delivering robust performance across complex problem distributions. Despite their success, a paradox remains: MetaBBO still rely on human-crafted Exploratory Landscape Analysis features to inform the meta-level agent about the low-level optimization progress. To address the gap, this paper proposes Neural Exploratory Landscape Analysis (NeurELA), a novel framework that dynamically profiles landscape features through a two-stage, attention-based neural network, executed in an entirely end-to-end fashion. NeurELA is pre-trained over a variety of MetaBBO algorithms using a multi-task neuroevolution strategy. Extensive experiments show that NeurELA achieves consistently superior performance when integrated into different and even unseen MetaBBO tasks and can be efficiently fine-tuned for further performance boost. This advancement marks a pivotal step in making MetaBBO algorithms more autonomous and broadly applicable. The source code of NeurELA can be accessed at https://github.com/GMC-DRL/Neur-ELA.

Figures (12)

• Figure 1: Left: The general workflow of MetaBBO algorithms, which follows a bi-level optimization paradigm. The landscape analyser timely profiles the low-level optimization progress, informing the meta-level neural policy to dynamically output desired configuration for the low-level BBO method. Right: The average optimization performance (larger is better) of integrating our pre-trained landscape analyser into diverse MetaBBO tasks (red line). The blue line denotes the original performance of the MetaBBO algorithms, while the green line denotes the performance of integrating the traditional exploratory landscape analysis features into the MetaBBO algorithms.
• Figure 1: The average wall time (in seconds) for computing features.
• Figure 2: Left: The architecture of the basic attention block ($Attn$). Right: The computation graph of the two-stage attention mechanism (Ts-Attn).
• Figure 3: Zero-shot performance of NeurELA on unseen MetaBBO algorithms and problem sets.
• Figure 4: Performance gain curves of fine-tuning NeurELA for a specific MetaBBO task.