A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving

TL;DR

The paper addresses the challenge of leveraging large language models to bridge informal problem descriptions and formal optimization, proposing a two-stage framework that separates optimization modeling from optimization solving. It provides a comprehensive taxonomy across modeling (prompt-based and learning-based) and solving (LLMs as optimizers, low-level LLMs, and high-level LLMs) paradigms, supported by a broad survey of applications and representative methods. Key findings reveal that while LLMs help with modeling and offer promising solving capabilities, they face data, cost, and reliability barriers, motivating dynamic, end-to-end, and agentic future systems. The work maps the field, identifies gaps, and outlines directions toward self-evolving optimization ecosystems that can operate across disciplines and real-world domains.

Abstract

Large Language Models (LLMs), with their strong understanding and reasoning capabilities, are increasingly being explored for tackling optimization problems, especially in synergy with evolutionary computation. While several recent surveys have explored aspects of LLMs for optimization, there remains a need for an integrative perspective that connects problem modeling with solving workflows. This survey addresses this gap by providing a comprehensive review of recent developments and organizing them within a structured framework. We classify existing research into two main stages: LLMs for optimization modeling and LLMs for optimization solving. The latter is further divided into three paradigms according to the role of LLMs in the optimization workflow: LLMs as stand-alone optimizers, low-level LLMs embedded within optimization algorithms, and high-level LLMs for algorithm selection and generation. For each category, we analyze representative methods, distill technical challenges, and examine their interplay with traditional approaches. We also review interdisciplinary applications spanning the natural sciences, engineering, and machine learning. By contrasting LLM-driven and conventional methods, we highlight key limitations and research gaps, and point toward future directions for developing self-evolving agentic ecosystems for optimization. An up-to-date collection of related literature is maintained at https://github.com/ishmael233/LLM4OPT.

Figures (6)

• Figure 1: Overall organization of this survey.
• Figure 2: Technological dependencies of LLMs for optimization, including EA paradigm and workflow, LLM architecture, and related enabling technologies.
• Figure 3: Illustration of LLMs for optimization modeling. Approaches can be broadly divided into two categories: (i) prompt-based methods, typically implemented via two-stage prompting, multi-agent collaboration, or interactive frameworks; and (ii) learning-based methods, which generally follow a workflow of data synthesis, model fine-tuning, and evaluation.
• Figure 4: Illustration of LLMs as optimizers. This paradigm primarily relies on interactive prompting for optimization, with a few studies incorporating fine-tuning and pre-training for enhanced performance.
• Figure 5: Illustration of low-level LLMs for optimization algorithms. LLMs can be applied at various stages within EAs, including initialization, evolutionary operators, algorithm configuration, and fitness evaluation.