Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap

TL;DR

This survey analyzes the bidirectional collaboration between large language models and evolutionary algorithms, outlining three main research paradigms: LLM-enhanced EA, EA-enhanced LLM, and applications driven by their synergy. It details how LLMs can act as black-box search operators and as sources for algorithm generation, while EAs can improve LLMs through prompt engineering and architecture search, among other capabilities. The paper surveys concrete methods and applications across code generation, software engineering, NAS, and other generative tasks, and provides a forward-looking roadmap addressing scalability, theory, and higher-level tasks. By mapping existing work and proposing future directions, it highlights the potential of LLMs and EAs to collectively advance optimization and AI capabilities in a diverse set of domains.

Abstract

Large language models (LLMs) have not only revolutionized natural language processing but also extended their prowess to various domains, marking a significant stride towards artificial general intelligence. The interplay between LLMs and evolutionary algorithms (EAs), despite differing in objectives and methodologies, share a common pursuit of applicability in complex problems. Meanwhile, EA can provide an optimization framework for LLM's further enhancement under black-box settings, empowering LLM with flexible global search capacities. On the other hand, the abundant domain knowledge inherent in LLMs could enable EA to conduct more intelligent searches. Furthermore, the text processing and generative capabilities of LLMs would aid in deploying EAs across a wide range of tasks. Based on these complementary advantages, this paper provides a thorough review and a forward-looking roadmap, categorizing the reciprocal inspiration into two main avenues: LLM-enhanced EA and EA-enhanced LLM. Some integrated synergy methods are further introduced to exemplify the complementarity between LLMs and EAs in diverse scenarios, including code generation, software engineering, neural architecture search, and various generation tasks. As the first comprehensive review focused on the EA research in the era of LLMs, this paper provides a foundational stepping stone for understanding the collaborative potential of LLMs and EAs. The identified challenges and future directions offer guidance for researchers and practitioners to unlock the full potential of this innovative collaboration in propelling advancements in optimization and artificial intelligence. We have created a GitHub repository to index the relevant papers: https://github.com/wuxingyu-ai/LLM4EC.

Figures (3)

• Figure 1: A general framework for integrated research of LLMs and EAs.
• Figure 2: Categorization of research works surveyed in this paper.
• Figure 3: Roadmap and Future Directions.