Large Language Models as Evolutionary Optimizers

TL;DR

EAs often rely on hand-crafted operators, which can be costly to design for new problems. This paper proposes LMEA, a zero-shot, LLM-driven evolutionary algorithm that uses prompts to guide the LLM in selecting parents and performing crossover/mutation, augmented by a self-adaptive temperature to balance exploration and exploitation. Empirical results on EUC-2D TSP instances (10–20 nodes showing optimal tours for 10/15) indicate that LMEA can match or surpass traditional heuristics and outperforms a baseline LL M-driven method, with self-adaptation contributing to faster convergence and better solutions. The work demonstrates the potential of LLMs as evolutionary optimizers and suggests directions for improving scalability, reducing runtime, and extending to other combinatorial tasks.

Abstract

Evolutionary algorithms (EAs) have achieved remarkable success in tackling complex combinatorial optimization problems. However, EAs often demand carefully-designed operators with the aid of domain expertise to achieve satisfactory performance. In this work, we present the first study on large language models (LLMs) as evolutionary combinatorial optimizers. The main advantage is that it requires minimal domain knowledge and human efforts, as well as no additional training of the model. This approach is referred to as LLM-driven EA (LMEA). Specifically, in each generation of the evolutionary search, LMEA instructs the LLM to select parent solutions from current population, and perform crossover and mutation to generate offspring solutions. Then, LMEA evaluates these new solutions and include them into the population for the next generation. LMEA is equipped with a self-adaptation mechanism that controls the temperature of the LLM. This enables it to balance between exploration and exploitation and prevents the search from getting stuck in local optima. We investigate the power of LMEA on the classical traveling salesman problems (TSPs) widely used in combinatorial optimization research. Notably, the results show that LMEA performs competitively to traditional heuristics in finding high-quality solutions on TSP instances with up to 20 nodes. Additionally, we also study the effectiveness of LLM-driven crossover/mutation and the self-adaptation mechanism in evolutionary search. In summary, our results reveal the great potentials of LLMs as evolutionary optimizers for solving combinatorial problems. We hope our research shall inspire future explorations on LLM-driven EAs for complex optimization challenges.

Figures (3)

• Figure 1: An overview of LMEA. The right half of this diagram demonstrates an example of the constructed prompt when utilizing LMEA to solve TSPs. The contents within "{}" in the prompt will be replaced with the corresponding input.
• Figure 2: Convergence curves: optimality gaps achieved by LMEA and OPRO as generation number increases. For each test set, the left figure illustrates the average optimality of the population, and the right figure illustrates optimality gap of the best found solution
• Figure 3: Convergence curves of LMEA and its variant without self-adaptation (LMEA*) on the rue-20 test set. (a) Optimality gap of the best found solution as the generation number increases. (b) Average optimality gap of the population as the generation number increases.