Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging Large Language Models to Develop Heuristics for Emerging Optimization Problems

Thomas Bömer, Nico Koltermann, Max Disselnmeyer, Laura Dörr, Anne Meyer

TL;DR

This study investigates whether LLMs can effectively generate heuristics for niche, not yet broadly researched optimization problems, using the unit-load pre-marshalling problem as an example case and proposes the Contextual Evolution of Heuristics (CEoH) framework, an extension of the Evolution of Heuristics (EoH) framework, which incorporates problem-specific descriptions to enhance in-context learning during heuristic generation.

Abstract

Combinatorial optimization problems often rely on heuristic algorithms to generate efficient solutions. However, the manual design of heuristics is resource-intensive and constrained by the designer's expertise. Recent advances in artificial intelligence, particularly large language models (LLMs), have demonstrated the potential to automate heuristic generation through evolutionary frameworks. Recent works focus only on well-known combinatorial optimization problems like the traveling salesman problem and online bin packing problem when designing constructive heuristics. This study investigates whether LLMs can effectively generate heuristics for niche, not yet broadly researched optimization problems, using the unit-load pre-marshalling problem as an example case. We propose the Contextual Evolution of Heuristics (CEoH) framework, an extension of the Evolution of Heuristics (EoH) framework, which incorporates problem-specific descriptions to enhance in-context learning during heuristic generation. Through computational experiments, we evaluate CEoH and EoH and compare the results. Results indicate that CEoH enables smaller LLMs to generate high-quality heuristics more consistently and even outperform larger models. Larger models demonstrate robust performance with or without contextualized prompts. The generated heuristics exhibit scalability to diverse instance configurations.

Leveraging Large Language Models to Develop Heuristics for Emerging Optimization Problems

TL;DR

This study investigates whether LLMs can effectively generate heuristics for niche, not yet broadly researched optimization problems, using the unit-load pre-marshalling problem as an example case and proposes the Contextual Evolution of Heuristics (CEoH) framework, an extension of the Evolution of Heuristics (EoH) framework, which incorporates problem-specific descriptions to enhance in-context learning during heuristic generation.

Abstract

Combinatorial optimization problems often rely on heuristic algorithms to generate efficient solutions. However, the manual design of heuristics is resource-intensive and constrained by the designer's expertise. Recent advances in artificial intelligence, particularly large language models (LLMs), have demonstrated the potential to automate heuristic generation through evolutionary frameworks. Recent works focus only on well-known combinatorial optimization problems like the traveling salesman problem and online bin packing problem when designing constructive heuristics. This study investigates whether LLMs can effectively generate heuristics for niche, not yet broadly researched optimization problems, using the unit-load pre-marshalling problem as an example case. We propose the Contextual Evolution of Heuristics (CEoH) framework, an extension of the Evolution of Heuristics (EoH) framework, which incorporates problem-specific descriptions to enhance in-context learning during heuristic generation. Through computational experiments, we evaluate CEoH and EoH and compare the results. Results indicate that CEoH enables smaller LLMs to generate high-quality heuristics more consistently and even outperform larger models. Larger models demonstrate robust performance with or without contextualized prompts. The generated heuristics exhibit scalability to diverse instance configurations.

Paper Structure

This paper contains 16 sections, 1 equation, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. The Unit-load Pre-marshalling Problem
  3. Related Work
  4. Related Work on Pre-marshalling
  5. Application of LLMs in Combinatorial Optimization
  6. Contextual Evolution of Heuristics Framework
  7. Evolutionary Procedure.
  8. Prompt Strategies
  9. Fitness function.
  10. Computational Experiments
  11. Parameters.
  12. Fitness.
  13. Generated heuristic code.
  14. Further Problem Instances.
  15. Conclusion
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Representation of a two-tier multibay block-stacking warehouse with three rows and columns per bay and four bays in total pfrommerboemer2024sorting. This configuration displays a 3x3 bay layout and 2x2 warehouse layout.
  • Figure 2: Example sequence of moves to solve an instance. Top-down view on a single-bay. Unit loads can be accessed from the north direction only.
  • Figure 3: The CEoH framework evolves code and thoughts while using an additional problem description.
  • Figure 4: Best found heuristic across generations for each model and experiment run. The best run for CEoH and EoH is shown with opacity. Lower values indicate better performance.
  • Figure 5: Heuristic thoughts and code with best fitness value for CEoH and EoH.