TIDE: Tuning-Integrated Dynamic Evolution for LLM-Based Automated Heuristic Design

TL;DR

The paper tackles automated heuristic design for NP-hard problems by addressing the coupling between discrete algorithmic structure and continuous parameters in LLM-driven design. It introduces TIDE, a nested evolution framework with an outer TSED-guided island model that preserves structural diversity and an inner co-evolution loop that combines UCB-based prompt strategy scheduling with a differential mutation operator for parameter tuning. Through extensive experiments across nine combinatorial optimization problems, TIDE consistently outperforms state-of-the-art baselines in solution quality while reducing token costs and improving search efficiency. This approach highlights the value of separating structure from parameters and leveraging adaptive, topology-aware migration to harness LLMs effectively for heuristic synthesis, with potential impact on scalable, cost-efficient automated design across domains.

Abstract

Although Large Language Models have advanced Automated Heuristic Design, treating algorithm evolution as a monolithic text generation task overlooks the coupling between discrete algorithmic structures and continuous numerical parameters. Consequently, existing methods often discard promising algorithms due to uncalibrated constants and suffer from premature convergence resulting from simple similarity metrics. To address these limitations, we propose TIDE, a Tuning-Integrated Dynamic Evolution framework designed to decouple structural reasoning from parameter optimization. TIDE features a nested architecture where an outer parallel island model utilizes Tree Similarity Edit Distance to drive structural diversity, while an inner loop integrates LLM-based logic generation with a differential mutation operator for parameter tuning. Additionally, a UCB-based scheduler dynamically prioritizes high-yield prompt strategies to optimize resource allocation. Extensive experiments across nine combinatorial optimization problems demonstrate that TIDE discovers heuristics that significantly outperform state-of-the-art baselines in solution quality while achieving improved search efficiency and reduced computational costs.

Figures (6)

• Figure 1: Left: Comparison of representative LLM-based AHD methods along Search Strategy (Stochastic $\to$ Systematic) and Heuristic Representation (Implicit $\to$ Explicit). Right: (a) Heuristic Representation: implicit heuristics lack explicit structure, whereas explicit heuristics define modular components for interpretability and structure. (b) Search Strategy: stochastic search uses random sampling, while systematic search employs guided mechanisms for efficient exploration.
• Figure 2: The pipeline of TIDE-AHD. Left:The outer loop functions as a TSED-guided Island Model, regulating global diversity via adaptive migration and fusion resets. Right: The inner loop executes co-evolutionary search, synergizing UCB-scheduled algorithmic logic generation with parameter refinement to mitigate numerical blindness.
• Figure 3: A key example on TSP50 under constructive framework: Convergence comparison of different LLM-based AHD methods. The x-axis denotes the number of evaluations, and the y-axis represents the objective value.
• Figure 4: Optimization Efficiency Analysis on TSP-50.
• Figure 5: Evolution of Structural Similarity (TSED) across islands.