Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving Integrated Process Planning and Scheduling Problem via Graph Neural Network Based Deep Reinforcement Learning

Hongpei Li, Han Zhang, Ziyan He, Yunkai Jia, Bo Jiang, Xiang Huang, Dongdong Ge

TL;DR

The paper tackles IPPS, an NP-hard problem that unifies process planning and shop scheduling. It introduces an end-to-end DRL framework that models IPPS as a Markov Decision Process and uses a heterogeneous graph neural network to capture complex relationships among operations, machines, and jobs, guided by PPO. Key contributions include proving MDP–IPPS equivalence (Theorem 1), introducing a combination-based action-space reduction to curb exploration, and designing a dense reward based on estimated end-times to stabilize learning. Empirical results on synthetic and public benchmarks show the approach achieves faster, higher-quality solutions than Greedy methods and remains competitive or superior to OR-Tools on large-scale instances, highlighting its potential for scalable, real-time IPPS in modern manufacturing. The work also demonstrates strong generalization capabilities and provides ablation evidence for the efficacy of reward design and graph representations.

Abstract

The Integrated Process Planning and Scheduling (IPPS) problem combines process route planning and shop scheduling to achieve high efficiency in manufacturing and maximize resource utilization, which is crucial for modern manufacturing systems. Traditional methods using Mixed Integer Linear Programming (MILP) and heuristic algorithms can not well balance solution quality and speed when solving IPPS. In this paper, we propose a novel end-to-end Deep Reinforcement Learning (DRL) method. We model the IPPS problem as a Markov Decision Process (MDP) and employ a Heterogeneous Graph Neural Network (GNN) to capture the complex relationships among operations, machines, and jobs. To optimize the scheduling strategy, we use Proximal Policy Optimization (PPO). Experimental results show that, compared to traditional methods, our approach significantly improves solution efficiency and quality in large-scale IPPS instances, providing superior scheduling strategies for modern intelligent manufacturing systems.

Solving Integrated Process Planning and Scheduling Problem via Graph Neural Network Based Deep Reinforcement Learning

TL;DR

The paper tackles IPPS, an NP-hard problem that unifies process planning and shop scheduling. It introduces an end-to-end DRL framework that models IPPS as a Markov Decision Process and uses a heterogeneous graph neural network to capture complex relationships among operations, machines, and jobs, guided by PPO. Key contributions include proving MDP–IPPS equivalence (Theorem 1), introducing a combination-based action-space reduction to curb exploration, and designing a dense reward based on estimated end-times to stabilize learning. Empirical results on synthetic and public benchmarks show the approach achieves faster, higher-quality solutions than Greedy methods and remains competitive or superior to OR-Tools on large-scale instances, highlighting its potential for scalable, real-time IPPS in modern manufacturing. The work also demonstrates strong generalization capabilities and provides ablation evidence for the efficacy of reward design and graph representations.

Abstract

The Integrated Process Planning and Scheduling (IPPS) problem combines process route planning and shop scheduling to achieve high efficiency in manufacturing and maximize resource utilization, which is crucial for modern manufacturing systems. Traditional methods using Mixed Integer Linear Programming (MILP) and heuristic algorithms can not well balance solution quality and speed when solving IPPS. In this paper, we propose a novel end-to-end Deep Reinforcement Learning (DRL) method. We model the IPPS problem as a Markov Decision Process (MDP) and employ a Heterogeneous Graph Neural Network (GNN) to capture the complex relationships among operations, machines, and jobs. To optimize the scheduling strategy, we use Proximal Policy Optimization (PPO). Experimental results show that, compared to traditional methods, our approach significantly improves solution efficiency and quality in large-scale IPPS instances, providing superior scheduling strategies for modern intelligent manufacturing systems.
Paper Structure (47 sections, 23 equations, 10 figures, 7 tables, 4 algorithms)

This paper contains 47 sections, 23 equations, 10 figures, 7 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Conventional IPPS Methods
  4. DRL-based Scheduling Methods
  5. Preliminaries
  6. Graph-based IPPS Problem
  7. Motivation and Novel Ingredient
  8. MDP Formulation
  9. State
  10. Action Space
  11. State Transitions
  12. Reward
  13. Policy
  14. Neural Networks and Decision Making
  15. Graph Representation
  16. ...and 32 more sections

Figures (10)

  • Figure 1: (a) AND/OR graph of an IPPS job. (b) All possible processing sequences for a single instance.
  • Figure 2: Comparison between IPPS (left) and Processing Planing plus Scheduling (right).
  • Figure 3: The workflow of the proposed method.
  • Figure 4: Example of heterogeneous graph.
  • Figure 5: Example of action space reduction strategy II.
  • ...and 5 more figures

Theorems & Definitions (1)

  • Remark 1