Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

G-LNS: Generative Large Neighborhood Search for LLM-Based Automatic Heuristic Design

Baoyun Zhao, He Wang, Liang Zeng

TL;DR

G-LNS tackles the structural bottleneck of LLM-based automated heuristic design by enabling LLMs to write both destroy and repair operators for Large Neighborhood Search. It maintains dual operator populations and evaluates their interaction with a synergy matrix, guiding co-evolution toward complementary logic. Across TSP, CVRP, and OVRP benchmarks, G-LNS achieves near-optimal solutions with reduced compute and demonstrates strong cross-distribution generalization to standard benchmarks. By shifting from evolving fixed templates to designing problem-specific, coupled operators, G-LNS delivers substantial performance gains and practical efficiency for challenging combinatorial optimization problems.

Abstract

While Large Language Models (LLMs) have recently shown promise in Automated Heuristic Design (AHD), existing approaches typically formulate AHD around constructive priority rules or parameterized local search guidance, thereby restricting the search space to fixed heuristic forms. Such designs offer limited capacity for structural exploration, making it difficult to escape deep local optima in complex Combinatorial Optimization Problems (COPs). In this work, we propose G-LNS, a generative evolutionary framework that extends LLM-based AHD to the automated design of Large Neighborhood Search (LNS) operators. Unlike prior methods that evolve heuristics in isolation, G-LNS leverages LLMs to co-evolve tightly coupled pairs of destroy and repair operators. A cooperative evaluation mechanism explicitly captures their interaction, enabling the discovery of complementary operator logic that jointly performs effective structural disruption and reconstruction. Extensive experiments on challenging COP benchmarks, such as Traveling Salesman Problems (TSP) and Capacitated Vehicle Routing Problems (CVRP), demonstrate that G-LNS significantly outperforms LLM-based AHD methods as well as strong classical solvers. The discovered heuristics not only achieve near-optimal solutions with reduced computational budgets but also exhibit robust generalization across diverse and unseen instance distributions.

G-LNS: Generative Large Neighborhood Search for LLM-Based Automatic Heuristic Design

TL;DR

G-LNS tackles the structural bottleneck of LLM-based automated heuristic design by enabling LLMs to write both destroy and repair operators for Large Neighborhood Search. It maintains dual operator populations and evaluates their interaction with a synergy matrix, guiding co-evolution toward complementary logic. Across TSP, CVRP, and OVRP benchmarks, G-LNS achieves near-optimal solutions with reduced compute and demonstrates strong cross-distribution generalization to standard benchmarks. By shifting from evolving fixed templates to designing problem-specific, coupled operators, G-LNS delivers substantial performance gains and practical efficiency for challenging combinatorial optimization problems.

Abstract

While Large Language Models (LLMs) have recently shown promise in Automated Heuristic Design (AHD), existing approaches typically formulate AHD around constructive priority rules or parameterized local search guidance, thereby restricting the search space to fixed heuristic forms. Such designs offer limited capacity for structural exploration, making it difficult to escape deep local optima in complex Combinatorial Optimization Problems (COPs). In this work, we propose G-LNS, a generative evolutionary framework that extends LLM-based AHD to the automated design of Large Neighborhood Search (LNS) operators. Unlike prior methods that evolve heuristics in isolation, G-LNS leverages LLMs to co-evolve tightly coupled pairs of destroy and repair operators. A cooperative evaluation mechanism explicitly captures their interaction, enabling the discovery of complementary operator logic that jointly performs effective structural disruption and reconstruction. Extensive experiments on challenging COP benchmarks, such as Traveling Salesman Problems (TSP) and Capacitated Vehicle Routing Problems (CVRP), demonstrate that G-LNS significantly outperforms LLM-based AHD methods as well as strong classical solvers. The discovered heuristics not only achieve near-optimal solutions with reduced computational budgets but also exhibit robust generalization across diverse and unseen instance distributions.
Paper Structure (50 sections, 9 equations, 4 figures, 8 tables, 1 algorithm)

This paper contains 50 sections, 9 equations, 4 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Automatic Heuristic Design
  4. Large Neighborhood Search
  5. Problem Formulation
  6. Related Work
  7. Methodology
  8. Overview of the G-LNS Framework
  9. Initialization
  10. Evaluation Phase
  11. Adaptive Selection.
  12. Scoring and Update.
  13. Population Management
  14. LLM-Driven Evolution Mechanism
  15. Experiments
  16. ...and 35 more sections

Figures (4)

  • Figure 1: Comparison of G-LNS and traditional AHD methods. For combinatorial optimization problems, unlike existing AHD methods that are largely restricted to local search, G-LNS enables structural reshaping through LLM-generated LNS operators, allowing the search to escape local optima.
  • Figure 2: The overall workflow of the G-LNS framework. The framework operates in a cyclic manner consisting of four phases: (1) Initialization, where the dual populations ($\mathcal{P}_d, \mathcal{P}_r$) are seeded with domain expertise and LLM-generated operators; (2) Evaluation, where operator pairs are dynamically selected and scored via an Adaptive LNS process; (3) Population Management, which ranks and prunes low-performing operators; and (4) Evolution, leveraging LLMs to perform mutation and crossover strategies to replenish the population with novel heuristics.
  • Figure 3: Convergence and Evolutionary Analysis.(a) Evolutionary Progress: Validation score trajectory of the best operator over 200 generations; the steady decline confirms the LLM's capacity to evolve high-performance heuristics. (b) Evaluation Progress: Convergence comparison on CVRP100 instances. G-LNS identifies the best solution in 70s across all 64 instances, significantly outperforming both the Solver (320s) and MCTS-AHD(ACO) (1110s) in terms of search efficiency.
  • Figure 4: Case Study on Structural Reshaping. Visualizing a snapshot of the evolutionary process on CVRP50. (a-b) The generated repair operator targets the entangled region for destruction. (c) The destroy operator resolves the crossing by optimizing node-to-vehicle assignments, reducing the cost from 11.26 to 9.96.