HGT-Scheduler: Deep Reinforcement Learning for the Job Shop Scheduling Problem via Heterogeneous Graph Transformers

TL;DR

The Heterogeneous Graph Transformer (HGT)-Scheduler is proposed, a reinforcement learning framework that models the JSSP as a heterogeneous graph and confirms that explicitly modeling distinct edge semantics improves the learning of effective scheduling policies.

Abstract

The Job Shop Scheduling Problem (JSSP) is commonly formulated as a disjunctive graph in which nodes represent operations and edges encode technological precedence constraints as well as machine-sharing conflicts. Most existing reinforcement learning approaches model this graph as homogeneous, merging job-precedence and machine-contention edges into a single relation type. Such a simplification overlooks the intrinsic heterogeneity of the problem structure and may lead to the loss of critical relational information. To address this limitation, we propose the Heterogeneous Graph Transformer (HGT)-Scheduler, a reinforcement learning framework that models the JSSP as a heterogeneous graph. The proposed architecture leverages a Heterogeneous Graph Transformer to capture type-specific relational patterns through edge-type-dependent attention mechanisms applied to precedence and contention relations. The scheduling policy is trained using Proximal Policy Optimization. The effectiveness of the proposed method is evaluated on the Fisher--Thompson benchmark instances. On the FT06 instance, the HGT-Scheduler achieves an optimality gap of 8.4\%, statistically outperforming both an identical architecture that ignores edge types ($p = 0.011$) and a standard Graph Isomorphism Network baseline. On the larger FT10 instance, the approach demonstrates favorable scalability. However, under a 50,000-step training limit, the performance of heterogeneous and homogeneous graph models is comparable, suggesting that edge-type awareness requires longer training horizons for larger problem instances. Ablation analyses further indicate that a three-layer attention architecture provides the best performance. Overall, the results confirm that explicitly modeling distinct edge semantics improves the learning of effective scheduling policies.

Figures (6)

• Figure 1: Main performance comparison across scheduling methods. The bar chart shows the mean makespan for each method on the FT06 and FT10 instances. Error bars indicate the standard deviation across five independent seeds. Lower makespan values indicate better performance. The dashed line represents the known optimal makespan for each instance.
• Figure 2: Optimality gap comparison. The chart displays the percentage deviation from the known optimal makespan. The HGT-Scheduler achieves a single-digit optimality gap on the FT06 instance, clearly separating itself from the homogeneous baselines.
• Figure 3: Makespan distribution across different scheduling methods for the FT06 and FT10 benchmark instances. The solid black line inside each box represents the median makespan. The dashed line represents the known optimal makespan.
• Figure 4: Learning curves tracking the mean makespan during the 50,000-step PPO training process. The solid lines represent the mean across five independent seeds. The shaded regions indicate the standard deviation. The HGT-Scheduler (blue line) demonstrates rapid, stable convergence on the FT06 instance. On FT10, all models exhibit high variance and incomplete convergence.
• Figure 5: Convergence of the optimality gap during training. This chart normalizes the learning curves relative to the known optimal makespan (0%). The HGT-Scheduler uniquely drives the optimality gap below 10% on the FT06 instance.