Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evolving Interdependent Operators with Large Language Models for Multi-Objective Combinatorial Optimization

Junhao Qiu, Xin Chen, Liang Ge, Liyong Lin, Zhichao Lu, Qingfu Zhang

TL;DR

This work formulates multi-operator design for MOEAs as a dynamic, interdependent optimization problem and introduces E2OC, a framework that co-evolves operator design strategies and executable code using LLM-generated guidance and Monte Carlo Tree Search. By modeling inter-operator coupling via a Granular design-space of design thoughts and applying an operator rotation mechanism, E2OC achieves superior performance over expert-designed operators and existing AHD methods on bi- and tri-objective FJSP and TSP benchmarks. The method demonstrates robust generalization across problem scales and objectives, and exhibits potential for continuous optimization through iterative re-use of elite strategies. These results highlight the value of explicit inter-operator coupling and structured search in automated algorithm design, with implications for scalable, human–AI collaborative optimization in complex MOO problems.

Abstract

Neighborhood search operators are critical to the performance of Multi-Objective Evolutionary Algorithms (MOEAs) and rely heavily on expert design. Although recent LLM-based Automated Heuristic Design (AHD) methods have made notable progress, they primarily optimize individual heuristics or components independently, lacking explicit exploration and exploitation of dynamic coupling relationships between multiple operators. In this paper, multi-operator optimization in MOEAs is formulated as a Markov decision process, enabling the improvement of interdependent operators through sequential decision-making. To address this, we propose the Evolution of Operator Combination (E2OC) framework for MOEAs, which achieves the co-evolution of design strategies and executable codes. E2OC employs Monte Carlo Tree Search to progressively search combinations of operator design strategies and adopts an operator rotation mechanism to identify effective operator configurations while supporting the integration of mainstream AHD methods as the underlying designer. Experimental results across AHD tasks with varying objectives and problem scales show that E2OC consistently outperforms state-of-the-art AHD and other multi-heuristic co-design frameworks, demonstrating strong generalization and sustained optimization capability.

Evolving Interdependent Operators with Large Language Models for Multi-Objective Combinatorial Optimization

TL;DR

This work formulates multi-operator design for MOEAs as a dynamic, interdependent optimization problem and introduces E2OC, a framework that co-evolves operator design strategies and executable code using LLM-generated guidance and Monte Carlo Tree Search. By modeling inter-operator coupling via a Granular design-space of design thoughts and applying an operator rotation mechanism, E2OC achieves superior performance over expert-designed operators and existing AHD methods on bi- and tri-objective FJSP and TSP benchmarks. The method demonstrates robust generalization across problem scales and objectives, and exhibits potential for continuous optimization through iterative re-use of elite strategies. These results highlight the value of explicit inter-operator coupling and structured search in automated algorithm design, with implications for scalable, human–AI collaborative optimization in complex MOO problems.

Abstract

Neighborhood search operators are critical to the performance of Multi-Objective Evolutionary Algorithms (MOEAs) and rely heavily on expert design. Although recent LLM-based Automated Heuristic Design (AHD) methods have made notable progress, they primarily optimize individual heuristics or components independently, lacking explicit exploration and exploitation of dynamic coupling relationships between multiple operators. In this paper, multi-operator optimization in MOEAs is formulated as a Markov decision process, enabling the improvement of interdependent operators through sequential decision-making. To address this, we propose the Evolution of Operator Combination (E2OC) framework for MOEAs, which achieves the co-evolution of design strategies and executable codes. E2OC employs Monte Carlo Tree Search to progressively search combinations of operator design strategies and adopts an operator rotation mechanism to identify effective operator configurations while supporting the integration of mainstream AHD methods as the underlying designer. Experimental results across AHD tasks with varying objectives and problem scales show that E2OC consistently outperforms state-of-the-art AHD and other multi-heuristic co-design frameworks, demonstrating strong generalization and sustained optimization capability.
Paper Structure (73 sections, 14 equations, 13 figures, 10 tables, 3 algorithms)

This paper contains 73 sections, 14 equations, 13 figures, 10 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Multi-Operator Optimization in MOEAs
  4. Multi-Objective Optimization.
  5. Multi-Operator Solver.
  6. Operator Combination Optimization.
  7. Markov Decision Process
  8. State Space.
  9. Action Space.
  10. Reward Function.
  11. Transition Probability.
  12. Methodology
  13. Overall Framework of E2OC
  14. Warm-Start Initialization of Multi-Operator Sets
  15. Language Space of Multi-level Thoughts
  16. ...and 58 more sections

Figures (13)

  • Figure 1: The design of single operators in MOEAs has evolved from reliance on (a) expert knowledge to (b) LLM-based iterative refinement of algorithmic ideas and code (e.g., EoH, MCTS_AHD). In contrast, (c) E2OC accounts for coupling among multiple operators and enables the co-evolution of design strategies and codes.
  • Figure 2: The framework of E2OC. Left: In the warm-start process, independently design operator sets and analyze improvement suggestions for different operators. Center: The language space of multi-operator strategies is complex. Prompts generated by differing design strategies among operators exhibit intricate coupling relationships. Right: Leverage branch selection and expansion in MCTS to explore advantageous combinations of operator design thoughts and strengthen dominant paths by operator rotation evolution.
  • Figure 3: Performance of different AHD methods on BIFJSP training (mk15) and test (mk14) instances.
  • Figure 4: Performance of different MCTS variants on BIFJSP training (mk15) and test (mk13) instances.
  • Figure 5: Prompt for design thought analysis and prompt rewriting.
  • ...and 8 more figures