LLM4AD: A Platform for Algorithm Design with Large Language Model

TL;DR

LLM4AD addresses the lack of a standardized, scalable toolkit for LLM-assisted algorithm design by introducing a unified Python platform with modular blocks for search methods, LLM interfaces, and task evaluation. It couples an iterative, population-based search with multi-objective capabilities and provides a robust evaluation sandbox, a diverse set of algorithm design tasks, examples, and a GUI to facilitate usage. Benchmark results across multiple tasks and eight LLMs demonstrate the value of combining search strategies with LLMs, while highlighting task-dependent variations and the absence of a single dominant model. The framework is designed for extensibility and fair comparison, aiming to accelerate innovation and standardization in the emerging field of LLM-based algorithm design.

Abstract

We introduce LLM4AD, a unified Python platform for algorithm design (AD) with large language models (LLMs). LLM4AD is a generic framework with modularized blocks for search methods, algorithm design tasks, and LLM interface. The platform integrates numerous key methods and supports a wide range of algorithm design tasks across various domains including optimization, machine learning, and scientific discovery. We have also designed a unified evaluation sandbox to ensure a secure and robust assessment of algorithms. Additionally, we have compiled a comprehensive suite of support resources, including tutorials, examples, a user manual, online resources, and a dedicated graphical user interface (GUI) to enhance the usage of LLM4AD. We believe this platform will serve as a valuable tool for fostering future development in the merging research direction of LLM-assisted algorithm design.

Figures (5)

• Figure 1:
• Figure 2: Graphical user interface (GUI) for LLM4AD.
• Figure 3: Convergence curve comparison on the performance (measured by fitness score) of the top-1 algorithms generated by GPT-4o-Mini. The performance averaged over three independent runs are denoted with markers (lower the better), while the standard deviations of scores are highlighted with the shaded regions.
• Figure 4: Convergence curve comparison on the performance of the top-1 heuristics generated by various LLMs. The mean score aggregated over three independent runs are denoted with markers (lower the better), while the standard deviations of scores are highlighted with the shaded regions.
• Figure 5: Radar plot on the performance of the top-1 algorithm generated by the EoH method using different LLMs. The radius of each vertex is calculated by the normalized fitness value over three independent runs; hence, a smaller radius/enclosed area indicates better performance.