irace-evo: Automatic Algorithm Configuration Extended With LLM-Based Code Evolution

TL;DR

irace-evo extends the irace automatic algorithm configuration framework to co-evolve parameters and algorithm code using LLM-driven edits, enabling multi-language support and token-efficient progressive context management. It introduces the Always-From-Original principle to ensure safe, bounded evolution by deriving new variants directly from the original variant $A^0$. In a CMSA for the Variable-Sized Bin Packing Problem, irace-evo discovers new placement heuristics that outperform the state-of-the-art CMSA, achieving competitive results with very low cost (under $€2$) by employing lightweight models such as Claude Haiku 3.5. These findings demonstrate that coupling automatic configuration with LLM-driven code evolution can yield robust, cost-efficient improvements in heuristic design and metaheuristic optimization.

Abstract

Automatic algorithm configuration tools such as irace efficiently tune parameter values but leave algorithmic code unchanged. This paper introduces a first version of irace-evo, an extension of irace that integrates code evolution through large language models (LLMs) to jointly explore parameter and code spaces. The proposed framework enables multi-language support (e.g., C++, Python), reduces token consumption via progressive context management, and employs the Always-From-Original principle to ensure robust and controlled code evolution. We evaluate irace-evo on the Construct, Merge, Solve & Adapt (CMSA) metaheuristic for the Variable-Sized Bin Packing Problem (VSBPP). Experimental results show that irace-evo can discover new algorithm variants that outperform the state-of-the-art CMSA implementation while maintaining low computational and monetary costs. Notably, irace-evo generates competitive algorithmic improvements using lightweight models (e.g., Claude Haiku 3.5) with a total usage cost under 2 euros. These results demonstrate that coupling automatic configuration with LLM-driven code evolution provides a powerful, cost-efficient avenue for advancing heuristic design and metaheuristic optimization.

Figures (3)

• Figure 1: A flowchart showing the integration of irace-evo into irace.
• Figure 2: Example of an irace-evo run with codeEvolutionVariants=2.
• Figure 3: Performance comparison of heuristics with and without independent tuning of parameters.