LLM Guided Evolution -- The Automation of Models Advancing Models

TL;DR

This work introduces Guided Evolution (GE), a framework that uses Large Language Models (LLMs) to directly mutate and mate neural network code, effectively reimagining neural architecture search as an LLM-guided evolutionary process. Central to GE are Evolution of Thought (EoT), which provides a feedback loop where the LLM reflects on prior mutations, and Character Role Play (CRP), which diversifies mutations via expert personas. The approach is demonstrated by evolving ExquisiteNetV2 on CIFAR10, yielding variants with higher accuracy (e.g., 93.34% from 92.52%) and, in some cases, substantial parameter reductions (e.g., 43.1% fewer parameters) while maintaining or improving performance. These results highlight the potential for autonomous model design and rapid exploration of architectural space, driven by intelligent prompts and self-improving reasoning, to accelerate NAS pipelines with human-like creativity and efficiency.

Abstract

In the realm of machine learning, traditional model development and automated approaches like AutoML typically rely on layers of abstraction, such as tree-based or Cartesian genetic programming. Our study introduces "Guided Evolution" (GE), a novel framework that diverges from these methods by utilizing Large Language Models (LLMs) to directly modify code. GE leverages LLMs for a more intelligent, supervised evolutionary process, guiding mutations and crossovers. Our unique "Evolution of Thought" (EoT) technique further enhances GE by enabling LLMs to reflect on and learn from the outcomes of previous mutations. This results in a self-sustaining feedback loop that augments decision-making in model evolution. GE maintains genetic diversity, crucial for evolutionary algorithms, by leveraging LLMs' capability to generate diverse responses from expertly crafted prompts and modulate model temperature. This not only accelerates the evolution process but also injects expert like creativity and insight into the process. Our application of GE in evolving the ExquisiteNetV2 model demonstrates its efficacy: the LLM-driven GE autonomously produced variants with improved accuracy, increasing from 92.52% to 93.34%, without compromising model compactness. This underscores the potential of LLMs to accelerate the traditional model design pipeline, enabling models to autonomously evolve and enhance their own designs.

Figures (7)

• Figure 1: LLM Driven Code Block Mating
• Figure 2: EoT prompt template
• Figure 3: Metrics of evolved ExquisiteNetV2 models.
• Figure 4: Evolved ExquisiteNetV2 models compared to State of the Art.
• Figure 5: Example of an augmented code block.