From Understanding to Excelling: Template-Free Algorithm Design through Structural-Functional Co-Evolution

TL;DR

The paper addresses the challenge of automating algorithm design end-to-end beyond template-based methods. It introduces a template-free framework that uses LLMs to translate natural-language requirements into code and employs a bi-dimensional co-evolution to optimize both function and architecture. The approach connects problem analysis, code generation, validation, and global optimization to automatically discover novel algorithms, as demonstrated on TSP/CVRP benchmarks and large-scale quadratic optimization problems, with results showing lower optimality gaps and faster convergence than baselines. By enabling holistic architectural innovation and autonomous search space expansion, the framework has the potential to significantly accelerate automated algorithm development and broaden the reach of LLMs in computational science.

Abstract

Large language models (LLMs) have greatly accelerated the automation of algorithm generation and optimization. However, current methods such as EoH and FunSearch mainly rely on predefined templates and expert-specified functions that focus solely on the local evolution of key functionalities. Consequently, they fail to fully leverage the synergistic benefits of the overall architecture and the potential of global optimization. In this paper, we introduce an end-to-end algorithm generation and optimization framework based on LLMs. Our approach utilizes the deep semantic understanding of LLMs to convert natural language requirements or human-authored papers into code solutions, and employs a two-dimensional co-evolution strategy to optimize both functional and structural aspects. This closed-loop process spans problem analysis, code generation, and global optimization, automatically identifying key algorithm modules for multi-level joint optimization and continually enhancing performance and design innovation. Extensive experiments demonstrate that our method outperforms traditional local optimization approaches in both performance and innovation, while also exhibiting strong adaptability to unknown environments and breakthrough potential in structural design. By building on human research, our framework generates and optimizes novel algorithms that surpass those designed by human experts, broadening the applicability of LLMs for algorithm design and providing a novel solution pathway for automated algorithm development.

Figures (3)

• Figure 1: Schematic Comparison of Algorithm Evolution: (a) Methods such as FunSearch/EOH primarily rely on predefined templates and iterate improvements at the level of individual functional modules through operations like crossover, mutation, reflection, and rewriting; (b) The proposed method leverages the deep semantic parsing capabilities of large language models to build a complete workflow, enabling multi-dimensional collaborative evolution of both algorithm structure and function. This approach breaks the limitations of single-function or local operations, achieving autonomous exploration and global optimization of the entire architecture, and providing adaptive, innovative solutions for complex problems.
• Figure 2: Performance comparison of five optimization algorithms—IQN, SLIQN, LISR-1, LISR-k, and the proposed LISR-k (CAE)—on optimization problems with increasing difficulty ($\kappa = 3.12 \times 10^6$, $3.12 \times 10^8$, $3.12 \times 10^{10}$). The x-axis represents time (s), and the y-axis shows the objective value. The results highlight that LISR-k (CAE) achieves faster convergence and better performance, particularly in highly challenging scenarios.
• Figure 3: Workflow comparison across three generations of algorithms: iter_num_0.py, iter_num_3.py, and iter_num_7.py. The blue regions represent areas with minor changes, while the red regions indicate significant modifications in the workflow. The figure illustrates the evolution from a basic implementation to an enhanced framework, emphasizing improvements in numerical stability, gradient correction, and adaptive scaling strategies.