Automatic Design of Optimization Test Problems with Large Language Models
Wojciech Achtelik, Hubert Guzowski, Maciej Smołka, Jacek Mańdziuk
TL;DR
The paper addresses the lack of diverse, interpretable benchmark functions for black-box optimization by introducing Evolution of Test Functions (EoTF), an LLM-driven evolutionary framework that synthesizes self-contained Python functions whose landscapes are tuned to a target Exploratory Landscape Analysis (ELA) profile $\boldsymbol{\phi}^*$. Through experiments on 24 BBOB functions and 24 MA-BBOB hybrids, EoTF demonstrates the ability to approximate target landscape properties and preserve optimizer rankings under fixed budgets, with stronger scalability to higher dimensions compared to NN-based generators. The study shows generalization to unseen problem instances and confirms that more capable LLMs yield incremental improvements, while maintaining interpretability of the generated benchmarks. Overall, EoTF offers a practical, portable, and scalable path to targeted benchmark generation that enhances the evaluation and development of black-box optimization methods.
Abstract
The development of black-box optimization algorithms depends on the availability of benchmark suites that are both diverse and representative of real-world problem landscapes. Widely used collections such as BBOB and CEC remain dominated by hand-crafted synthetic functions and provide limited coverage of the high-dimensional space of Exploratory Landscape Analysis (ELA) features, which in turn biases evaluation and hinders training of meta-black-box optimizers. We introduce Evolution of Test Functions (EoTF), a framework that automatically generates continuous optimization test functions whose landscapes match a specified target ELA feature vector. EoTF adapts LLM-driven evolutionary search, originally proposed for heuristic discovery, to evolve interpretable, self-contained numpy implementations of objective functions by minimizing the distance between sampled ELA features of generated candidates and a target profile. In experiments on 24 noiseless BBOB functions and a contamination-mitigating suite of 24 MA-BBOB hybrid functions, EoTF reliably produces non-trivial functions with closely matching ELA characteristics and preserves optimizer performance rankings under fixed evaluation budgets, supporting their validity as surrogate benchmarks. While a baseline neural-network-based generator achieves higher accuracy in 2D, EoTF substantially outperforms it in 3D and exhibits stable solution quality as dimensionality increases, highlighting favorable scalability. Overall, EoTF offers a practical route to scalable, portable, and interpretable benchmark generation targeted to desired landscape properties.