ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution

TL;DR

ReEvo introduces Language Hyper-Heuristics (LHHs) that use large language models to generate heuristics and pairs them with Reflective Evolution to guide search via verbal reflections. The framework uses a two-LLM agent setup (generator and reflector) within an evolutionary loop to produce code-based heuristics, achieving SOTA or competitive results across six combinatorial optimization problems and five algorithmic types with high sample efficiency. Fitness landscape analysis and black-box prompting underpin reliable evaluation, while ablations and comparisons to EoH demonstrate the value of dual-level reflections for both inference and optimization. The work highlights the potential of LHHs to broaden heuristic design, improve interpretability, and extend to real-world optimization tasks where evaluation costs are high.

Abstract

The omnipresence of NP-hard combinatorial optimization problems (COPs) compels domain experts to engage in trial-and-error heuristic design. The long-standing endeavor of design automation has gained new momentum with the rise of large language models (LLMs). This paper introduces Language Hyper-Heuristics (LHHs), an emerging variant of Hyper-Heuristics that leverages LLMs for heuristic generation, featuring minimal manual intervention and open-ended heuristic spaces. To empower LHHs, we present Reflective Evolution (ReEvo), a novel integration of evolutionary search for efficiently exploring the heuristic space, and LLM reflections to provide verbal gradients within the space. Across five heterogeneous algorithmic types, six different COPs, and both white-box and black-box views of COPs, ReEvo yields state-of-the-art and competitive meta-heuristics, evolutionary algorithms, heuristics, and neural solvers, while being more sample-efficient than prior LHHs.

Figures (4)

• Figure 1: An illustration of ReEvo.
• Figure 2: Comparative evaluations of ACO using expert-designed heuristics skinderowicz2022_aco_tsp_human_blcai2022_aco_cvrp_human_blsohrabi2021_aco_op_human_blfidanova2020_aco_mkp_human_bllevine2004_aco_bpp_human_bl, neural heuristics ye2023deepaco, and ReEvo heuristics. For each COP, the same neural heuristic or the ReEvo heuristic is applied across all problem sizes; both heuristics are trained exclusively on the smallest problem size among the five. Left: Relative performance improvement of DeepACO and ReEvo over human baselines w.r.t. problem sizes. Right: ACO evolution curves, plotting the all-time best objective value w.r.t. the number of solution evaluations. The curves are averaged over three runs in which only small variances are observed (e.g., $\sim 0.01$ for TSP50).
• Figure 3: Left: Comparison of DevFormer kim2023devformer, the expert-designed GA park2023versatile and our ReEvo-designed GA on DPP. The evolution curves plot the best objective value over generations; the horizontal line indicates the reward of end-to-end solutions generated by DevFormer. Right: Evaluation results of DPP solvers. We report the number of solution generations and the average objective value of 100 test problems.
• Figure 4: Comparisons between EoH liu2024_llm_gls and ReEvo on five COPs with black-box prompting and using different LLMs. We perform three runs for each setting.

Theorems & Definitions (5)

• Definition 3.1: Hyper-Heuristic
• Definition 3.2: Language Hyper-Heuristic
• Definition 6.1: Neighborhood
• Definition 6.2: Autocorrelation
• Definition 6.3: Correlation Length