Automated Design of Metaheuristic Algorithms: A Survey

TL;DR

This survey addresses the automated design of metaheuristic algorithms by introducing a four module taxonomy that structures the design space, strategies, performance evaluation, and target problems. It consolidates methods across design spaces based on computational primitives and algorithmic operators, and surveys model free and model based design strategies including GP, grammars, Bayesian optimization, reinforcement learning, and LLMs. The report reviews performance metrics, evaluation protocols, and cost reduction techniques, and discusses numerical benchmarks, practical problems, and software platforms (eg, irace, ParamILS, SMAC). It also identifies research trends and calls for deeper experimental/theoretical analyses and broader real world deployments to advance autonomous algorithm design. The work aims to democratize access to high performance metaheuristics and catalyze progress toward autonomous and general AI through systematic synthesis and guidance.

Abstract

Metaheuristics have gained great success in academia and practice because their search logic can be applied to any problem with available solution representation, solution quality evaluation, and certain notions of locality. Manually designing metaheuristic algorithms for solving a target problem is criticized for being laborious, error-prone, and requiring intensive specialized knowledge. This gives rise to increasing interest in automated design of metaheuristic algorithms. With computing power to fully explore potential design choices, the automated design could reach and even surpass human-level design and could make high-performance algorithms accessible to a much wider range of researchers and practitioners. This paper presents a broad picture of automated design of metaheuristic algorithms, by conducting a survey on the common grounds and representative techniques in terms of design space, design strategies, performance evaluation strategies, and target problems in this field.

Figures (7)

• Figure 1: Abstractive process of automated design of metaheuristic algorithms.
• Figure 2: Example of a metaheuristic operator represented by a binary GP-style tree. This example represents the whole arithmetic crossover operator eiben2003introduction, where $\rm{x_1}$ and $\rm{x_2}$ are the input variables (i.e., two parent solutions), $w$ is a weight parameter, and $+,*,-$ are primitives. The operator performs $w*\rm{x_1}+(1-w)*\rm{x_2}$ in mathematical form.
• Figure 3: Example of a metaheuristic operator represented by a linear array of computational primitives. The example is derived from goldman2011self. It represents a crossover operator consisting of a $swap$ and $merge$ primitives. The indexes and parameters associated with the primitives are in brackets. This operator produces offspring by first swapping the 3rd element of parent 1 and the 5th element of parent 2, then merging the 1st element of parent 1 (by a weight of 0.7) with a random element of parent 2.
• Figure 4: Example of grammar for designing genetic algorithms. In (a), the production rule $\texttt{<initialization>}$ means that the initialization operator can be either $\texttt{uniform random}$ or $\texttt{Latin hypercube sample}$; the meaning of other rules is in the same fashion. According to the $\texttt{<algorithm>}$ rule, two elitism GA structures can be represented: one searches by crossover and mutation; the other searches by mutation only. An algorithm instantiated by the rules is shown in (b).
• Figure 5: Example of a metaheuristic algorithm represented by a directed acyclic graph. The example is derived from ryser2016iterative. Nodes indexed with $1,2,3,4$ represent algorithmic operators. "$(0)$" for node $1$ means that node $1$ connects from node $0$; the meaning for other nodes is in the same fashion. The exemplified algorithm consists of one input, one output, and three operators $1,2,3$. Operator $4$ is inactivated because it is not on the path from the input to the output.