Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TransGP: Task-Conditioned Transformer-Guided Genetic Programming for Multitask Dynamic Flexible Job Shop Scheduling

Meng Xu, Jiao Liu, Hua Yu, Yew Soon Ong

Abstract

Hyper-heuristics have become a popular approach for solving dynamic flexible job shop scheduling (DFJSS) problems. They use gradient-free optimization techniques like Genetic Programming (GP) to evolve non-differentiable heuristics. However, conventional GP methods tend to converge slowly because they rely solely on evolutionary search to find good heuristics. Existing multitask GP methods can solve multiple tasks simultaneously and speed up the search by transferring knowledge across similar tasks. But they mostly exchange heuristic building blocks without truly generating heuristics conditioned on task information. In this paper, we aim to accelerate convergence and enable task-specific heuristic generation by incorporating a task-conditioned Transformer model. The Transformer works in two ways. First, it learns the distribution of elite heuristics, biasing the search toward promising regions of the heuristic space. Second, through conditional generation, it produces heuristics tailored to specific tasks, allowing the model to handle multiple scheduling tasks at once and improving overall optimization efficiency. Based on these ideas, we propose TransGP, a Task-Conditioned Transformer-Guided GP framework. This evolutionary paradigm integrates generative modeling with GP, enabling efficient multitask heuristic learning and knowledge transfer. We evaluate TransGP on a range of DFJSS scenarios. Experimental results show that TransGP consistently outperforms multitask GP baselines, widely used handcrafted heuristics, and the pure Transformer model, achieving faster convergence, superior solution quality, and enhanced robustness.

TransGP: Task-Conditioned Transformer-Guided Genetic Programming for Multitask Dynamic Flexible Job Shop Scheduling

Abstract

Hyper-heuristics have become a popular approach for solving dynamic flexible job shop scheduling (DFJSS) problems. They use gradient-free optimization techniques like Genetic Programming (GP) to evolve non-differentiable heuristics. However, conventional GP methods tend to converge slowly because they rely solely on evolutionary search to find good heuristics. Existing multitask GP methods can solve multiple tasks simultaneously and speed up the search by transferring knowledge across similar tasks. But they mostly exchange heuristic building blocks without truly generating heuristics conditioned on task information. In this paper, we aim to accelerate convergence and enable task-specific heuristic generation by incorporating a task-conditioned Transformer model. The Transformer works in two ways. First, it learns the distribution of elite heuristics, biasing the search toward promising regions of the heuristic space. Second, through conditional generation, it produces heuristics tailored to specific tasks, allowing the model to handle multiple scheduling tasks at once and improving overall optimization efficiency. Based on these ideas, we propose TransGP, a Task-Conditioned Transformer-Guided GP framework. This evolutionary paradigm integrates generative modeling with GP, enabling efficient multitask heuristic learning and knowledge transfer. We evaluate TransGP on a range of DFJSS scenarios. Experimental results show that TransGP consistently outperforms multitask GP baselines, widely used handcrafted heuristics, and the pure Transformer model, achieving faster convergence, superior solution quality, and enhanced robustness.

Paper Structure

This paper contains 37 sections, 11 equations, 10 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multitask Dynamic Flexible Job Shop Scheduling
  4. Symbolic Heuristics for Scheduling
  5. Genetic Programming for Multitask Scheduling
  6. Generative Models for Symbolic Expression Learning
  7. Method
  8. Overall Framework
  9. Task-Conditioned Transformer Model for Symbolic Heuristic Generation
  10. Vectorization of Symbolic Heuristics
  11. Task-Conditioned Transformer Training
  12. Task-Conditioned Transformer-Guided GP
  13. Task-Conditioned Transformer-Guided Mutation
  14. Computational Cost Discussion
  15. Experiment Design
  16. ...and 22 more sections

Figures (10)

  • Figure 1: An example of a symbolic heuristic in DFJSS, which contains complex and irregular structures.
  • Figure 2: Overall framework of TransGP.
  • Figure 3: Vectorization of symbolic heuristics.
  • Figure 4: Overview of the evolutionary framework in TransGP and an example of Transformer-guided mutation process.
  • Figure 5: Convergence curves of test performance across 30 independent runs for TransGP and baseline GP methods on three scenarios, each containing three tasks.
  • ...and 5 more figures