Discovering Heuristics with Large Language Models (LLMs) for Mixed-Integer Programs: Single-Machine Scheduling

TL;DR

The paper addresses SMTT, an NP-hard single-machine scheduling problem, by leveraging an LLM-driven, island-based evolutionary framework to discover new heuristics. It introduces two algorithms, EDD Challenger (EDDC) and MDD Challenger (MDDC), built on EDD and MDD foundations and evaluated against exact methods (MIP/DP) and established heuristics. Through rigorous testing on 20–500 job instances, augmented MDDC consistently achieves the smallest optimality gaps and favorable runtimes, demonstrating robust scalability and practical applicability. The work highlights the value of human-LLM collaboration, prompt engineering, and a transparent, reproducible pipeline for discovering high-performance heuristics for NP-hard problems, with potential extensions to broader combinatorial optimization tasks. The key mathematical formulations underpinning SMTT, including the MIP with positional assignments ($1 \| \sum T_j$) and tardiness $T_j = \max\{0, C_j - d_j\}$, ground the study in established theory while the LLM-driven methods push practical performance beyond traditional heuristics on large instances.

Abstract

Our study contributes to the scheduling and combinatorial optimization literature with new heuristics discovered by leveraging the power of Large Language Models (LLMs). We focus on the single-machine total tardiness (SMTT) problem, which aims to minimize total tardiness by sequencing n jobs on a single processor without preemption, given processing times and due dates. We develop and benchmark two novel LLM-discovered heuristics, the EDD Challenger (EDDC) and MDD Challenger (MDDC), inspired by the well-known Earliest Due Date (EDD) and Modified Due Date (MDD) rules. In contrast to prior studies that employed simpler rule-based heuristics, we evaluate our LLM-discovered algorithms using rigorous criteria, including optimality gaps and solution time derived from a mixed-integer programming (MIP) formulation of SMTT. We compare their performance against state-of-the-art heuristics and exact methods across various job sizes (20, 100, 200, and 500 jobs). For instances with more than 100 jobs, exact methods such as MIP and dynamic programming become computationally intractable. Up to 500 jobs, EDDC improves upon the classic EDD rule and another widely used algorithm in the literature. MDDC consistently outperforms traditional heuristics and remains competitive with exact approaches, particularly on larger and more complex instances. This study shows that human-LLM collaboration can produce scalable, high-performing heuristics for NP-hard constrained combinatorial optimization, even under limited resources when effectively configured.

Figures (7)

• Figure 1: Illustration of the heuristic discovery pipeline (inspired by funsearch). Figure created by the authors.
• Figure 2: Comparison of our study and FunSearch (funsearch)
• Figure 3: Average Total Tardiness Throughout the Training for EDDC Over Iterations
• Figure 4: Average Total Tardiness Throughout the Training for MDDC Over Iterations
• Figure A1: 'Evaluate' function for SMTT problem