PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMs

TL;DR

PathWise reframes automated heuristic design for COPs as stateful, plan-driven reasoning over an entailment graph, guided by a policy agent, a world model, and rhetorical critics. This two-timescale framework enables memory of derivations, diverse exploration, and critique-informed refinement, yielding faster convergence and stronger generalization across multiple COPs and LLM backbones. Empirical results across TSP, KP, CVRP, MKP, OP, and BPP demonstrate superior performance to existing LLM-based AHD methods and robustness to backbone differences, with ablations confirming the value of critics and prompt diversity. The work advances scalable, memory-aware AHD, reducing manual heuristic engineering and enabling broader applicability in real-world optimization tasks.

Abstract

Large Language Models (LLMs) have enabled automated heuristic design (AHD) for combinatorial optimization problems (COPs), but existing frameworks' reliance on fixed evolutionary rules and static prompt templates often leads to myopic heuristic generation, redundant evaluations, and limited reasoning about how new heuristics should be derived. We propose a novel multi-agent reasoning framework, referred to as Planning through World Model for Automated Heuristic Design via Self-Evolving LLMs (PathWise), which formulates heuristic generation as a sequential decision process over an entailment graph serving as a compact, stateful memory of the search trajectory. This approach allows the system to carry forward past decisions and reuse or avoid derivation information across generations. A policy agent plans evolutionary actions, a world model agent generates heuristic rollouts conditioned on those actions, and critic agents provide routed reflections summarizing lessons from prior steps, shifting LLM-based AHD from trial-and-error evolution toward state-aware planning through reasoning. Experiments across diverse COPs show that PathWise converges faster to better heuristics, generalizes across different LLM backbones, and scales to larger problem sizes.

Figures (5)

• Figure 1: Evolution curves of LLM-based AHD methods on (a) TSP and (b) CVRP under different search frameworks, showing best-so-far heuristic performance as a function of evaluation number for representative population-based and tree-based methods using GPT-4o-mini. PathWise is run with a limit of $n_e = 500$ evaluations while all baselines use $n_e = 1000$, yet achieves stronger performance with lower variance and faster convergence.
• Figure 2: Overview of PathWise. (a) AHD is orchestrated across two timescales, with inner entailment steps within each outer iteration; example shown with $N_p = 4$ (parent metadata in nodes omitted for simplicity). (b) Entailment step showing policy and world model interaction with critics, carrying forward lessons to guide heuristic generation; illustrated at $t = 1$ resulting in the entailment of $v_6$.
• Figure 3: Evolution curves of LLM-based AHD methods using GPT-5-nano (low) on (a) CVRP and GPT-5-nano (medium) on (b) MKP, with a limit of $n_e = 500$ heuristic evaluations. Each curve is averaged over the 3 runs used in Table \ref{['tab:aco_general_framework_merged']}.
• Figure 4: Example of PathWise at outer iteration $r=1$ and entailment step $t=2$, showing the entailed node $v_\star$. The current state $s_t$ contains nodes $\{v_1,\dots,v_5\}$. The policy agent $\pi_p$ selects a parent set and generates a derivation rationale, which is executed by the world model $\pi_{wm}$ to entail a new node. The resulting entailed node is $v_\star$ with $i_\star=0$ and $j_\star=1$, as shown. The policy and world model critics provide routed reflections at step $t$, while shaded boxes indicate the updated reflections at step $t+1$, conditioning the next step.
• Figure 5: Example of PathWise at outer iteration $r=34$ and entailment step $t=1$, showing the entailed node $v_\star$. The current state $s_t$ contains nodes $\{v_1,\dots,v_4\}$. The policy agent $\pi_p$ selects a parent set and generates a derivation rationale, which is executed by the world model $\pi_{wm}$ to entail a new node. The resulting entailed node is $v_\star$ with $i_\star=1$ and $j_\star=1$, as shown. The policy and world model critics provide routed reflections at step $t$, while shaded boxes indicate the updated reflections at step $t+1$, conditioning the next step.