Game-Theoretic Co-Evolution for LLM-Based Heuristic Discovery

TL;DR

ASRO reframes automatic heuristic design as a program-space, two-player zero-sum game between solvers and instance generators, maintaining persistent strategy pools and using LLM-based approximate best-response oracles to create a self-generated curriculum. By replacing static evaluation with an adaptive co-evolutionary loop, ASRO achieves stronger generalization and robustness across online bin packing, TSP, and CVRP, outperforming static or single-agent baselines while incurring extra compute that is amenable to parallelization. The framework is oracle-agnostic with respect to the underlying BR search and demonstrates that persistent co-adaptation can uncover solver strategies that generalize to diverse and structured instance distributions. The work lays a principled foundation for co-evolving executable solver and instance-generation programs and suggests promising extensions to multi-objective, teacher–student, and broader algorithmic reasoning contexts.

Abstract

Large language models (LLMs) have enabled rapid progress in automatic heuristic discovery (AHD), yet most existing methods are predominantly limited by static evaluation against fixed instance distributions, leading to potential overfitting and poor generalization under distributional shifts. We propose Algorithm Space Response Oracles (ASRO), a game-theoretic framework that reframes heuristic discovery as a program level co-evolution between solver and instance generator. ASRO models their interaction as a two-player zero-sum game, maintains growing strategy pools on both sides, and iteratively expands them via LLM-based best-response oracles against mixed opponent meta-strategies, thereby replacing static evaluation with an adaptive, self-generated curriculum. Across multiple combinatorial optimization domains, ASRO consistently outperforms static-training AHD baselines built on the same program search mechanisms, achieving substantially improved generalization and robustness on diverse and out-of-distribution instances.

Figures (13)

• Figure 1: Overview of ASRO. The iterative process is structured into three phases. (Left) The current solver strategy pool $\mathcal{S}^{(t)}$ and generator strategy pool $\mathcal{G}^{(t)}$ are evaluated pairwise to compute a payoff matrix $M^{(t)}$. A meta-solver then computes mixed meta-strategies $(\sigma_s^{(t)}, \sigma_g^{(t)})$ over the existing strategy pools. (Middle) Program-space best-response oracles synthesize new solver and generator programs conditioned on the opponent’s current meta-strategy, instantiated by an underlying program search mechanism (see Figure \ref{['fig:eoh_oracle']}). (Right) The newly discovered solver $s_{t+1}$ and generator $g_{t+1}$ are added to their respective strategy pools for the next iteration ($t \to t+1$), enabling systematic exploration of the program space.
• Figure 2: Program-space best-response oracle in ASRO, instantiated via an EoH-style evolutionary program search. Under a fixed opponent meta-strategy $\sigma_{-i}$ provided by ASRO, the oracle performs a bounded evolutionary search over programs in the corresponding strategy space. Candidate programs are initialized, explored, and mutated using structured LLM-based operators, and evaluated according to the induced best-response objective.
• Figure 3: ASRO training dynamics across combinatorial optimization tasks. (Top) Convergence behavior of ASRO on OBP, TSP, and CVRP, measured by solver exploitability, generator exploitability, and ANC over iterations, shown for a representative ASRO run; shaded regions denote confidence intervals via bootstrap resampling over instances (20 per generator). (Bottom) Stability across random runs: test gap (%) (mean $\pm$ std) over five independent runs on the hardest benchmark for each task, where benchmark difficulty is determined by the EoH baseline ranking.
• Figure 4: A snapshot of solver--generator co-evolution in online bin packing. The figure shows a utility matrix at Round 5, with meta-strategies computed from Round 4. Side panels summarize the solver and generator strategies present in the pool at Round 4. In this snapshot, the generator-side meta-strategy assigns dominant weight to $g_4$, and the solver $s_5$ is generated as a best response under this adversarial pressure. The dashed connection indicates that $s_5$ is generated as a best response to $g_4$. The highlighted entry (red box) shows the utility of $s_5$ evaluated against $g_4$.
• Figure 5: Unified structure of all prompt templates used by EoH evolutionary operators. Each operator (I1, E1, E2, M1, M2, M3) uses this five-part template with different operator-specific instructions.