Investigating the Monte-Carlo Tree Search Approach for the Job Shop Scheduling Problem
Laurie Boveroux, Damien Ernst, Quentin Louveaux
TL;DR
This work tackles the large-scale JSSP with recirculation by formulating multiple MDPs and solving them with Monte Carlo Tree Search (MCTS). It introduces a new synthetic benchmark derived from anonymised real manufacturing data and compares MCTS against a constraint programming baseline, showing MCTS to yield high-quality solutions for large instances. Key contributions include diverse MDP formulations, a realistic benchmark, and empirical evidence that idle-time insertion strategies improve MCTS performance. The findings highlight MCTS as a scalable and effective heuristic for industrial JSSP, with potential for further gains via learned reward functions for partial schedules.
Abstract
The Job Shop Scheduling Problem (JSSP) is a well-known optimization problem in manufacturing, where the goal is to determine the optimal sequence of jobs across different machines to minimize a given objective. In this work, we focus on minimising the weighted sum of job completion times. We explore the potential of Monte Carlo Tree Search (MCTS), a heuristic-based reinforcement learning technique, to solve large-scale JSSPs, especially those with recirculation. We propose several Markov Decision Process (MDP) formulations to model the JSSP for the MCTS algorithm. In addition, we introduce a new synthetic benchmark derived from real manufacturing data, which captures the complexity of large, non-rectangular instances often encountered in practice. Our experimental results show that MCTS effectively produces good-quality solutions for large-scale JSSP instances, outperforming our constraint programming approach.