Learning Flexible Job Shop Scheduling under Limited Buffers and Material Kitting Constraints

TL;DR

Experimental results show that the proposed method outperforms traditional heuristics and advanced DRL methods in terms of makespan and pallet changes, and also achieves a good balance between solution quality and computational cost.

Abstract

The Flexible Job Shop Scheduling Problem (FJSP) originates from real production lines, while some practical constraints are often ignored or idealized in current FJSP studies, among which the limited buffer problem has a particular impact on production efficiency. To this end, we study an extended problem that is closer to practical scenarios--the Flexible Job Shop Scheduling Problem with Limited Buffers and Material Kitting. In recent years, deep reinforcement learning (DRL) has demonstrated considerable potential in scheduling tasks. However, its capacity for state modeling remains limited when handling complex dependencies and long-term constraints. To address this, we leverage a heterogeneous graph network within the DRL framework to model the global state. By constructing efficient message passing among machines, operations, and buffers, the network focuses on avoiding decisions that may cause frequent pallet changes during long-sequence scheduling, thereby helping improve buffer utilization and overall decision quality. Experimental results on both synthetic and real production line datasets show that the proposed method outperforms traditional heuristics and advanced DRL methods in terms of makespan and pallet changes, and also achieves a good balance between solution quality and computational cost. Furthermore, a supplementary video is provided to showcase a simulation system that effectively visualizes the progression of the production line.

Figures (8)

• Figure 1: Comparison of the makespan, pallet changes, and computation time (the closer to the bottom and left, the better). Our method establishes an effective balance between solution quality and computational cost.
• Figure 2: FJSP with limited buffer and material kitting constraints. Green, white, and yellow circles indicate scheduled, unscheduled, and currently being scheduled operations. Parts of the same job share color; identical shapes denote the same part type. Operation $O_{22}$ is subject to limited buffer and kitting constraints: (1) Six pallets are available, each preloaded with parts; (2) Job $J_2$'s parts must be split across two pallets, but one type is not in existing categories; (3) A pallet must be replaced with an empty one; (4) Once an empty pallet is available, all parts can be properly assigned.
• Figure 3: Example of state transition. At timestep $t$, both unscheduled operations $O_{32}$ and $O_{22}$ are subject to buffer constraints; therefore, the buffer node is connected to both of them. The algorithm then prepares to execute the $O_{22}-M_{2}$ action. Upon execution, the graph transitions to state $s_{t+1}$. The newly scheduled operation node is marked green, and its associated connections are updated accordingly.
• Figure 4: Heterogeneous GNN uses multiple state features as input, and outputs the O-M pair decision.
• Figure 5: Message passing between the buffer and the part-sorting operations, $w$ is the weight of the edge