Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Optimize Job Shop Scheduling Under Structural Uncertainty

Rui Zhang, Jianwei Niu, Xuefeng Liu, Shaojie Tang, Jing Yuan

TL;DR

This work tackles the Job-Shop Scheduling Problem under structural uncertainty, where routing paths are determined by uncertain, scenario-driven factors. It introduces UP-AAC, an Asymmetric Actor-Critic framework that trains the Critic on deterministic hindsight states while the Actor operates on stochastic states, enabling low-variance, stable policy updates; paired with the Uncertainty Perception Model (UPM) that injects a global risk representation into decisions. Empirically, UP-AAC achieves state-of-the-art makespan performance and robustness (CVaR) across small, medium, and large instances, with ablations showing both AAC and UPM are essential for gains. The approach offers a practical pathway to robust, uncertainty-aware scheduling in complex manufacturing settings and suggests extensions to leverage historical data for continual improvement.

Abstract

The Job-Shop Scheduling Problem (JSSP), under various forms of manufacturing uncertainty, has recently attracted considerable research attention. Most existing studies focus on parameter uncertainty, such as variable processing times, and typically adopt the actor-critic framework. In this paper, we explore a different but prevalent form of uncertainty in JSSP: structural uncertainty. Structural uncertainty arises when a job may follow one of several routing paths, and the selection is determined not by policy, but by situational factors (e.g., the quality of intermediate products) that cannot be known in advance. Existing methods struggle to address this challenge due to incorrect credit assignment: a high-quality action may be unfairly penalized if it is followed by a time-consuming path. To address this problem, we propose a novel method named UP-AAC. In contrast to conventional actor-critic methods, UP-AAC employs an asymmetric architecture. While its actor receives a standard stochastic state, the critic is crucially provided with a deterministic state reconstructed in hindsight. This design allows the critic to learn a more accurate value function, which in turn provides a lower-variance policy gradient to the actor, leading to more stable learning. In addition, we design an attention-based Uncertainty Perception Model (UPM) to enhance the actor's scheduling decisions. Extensive experiments demonstrate that our method outperforms existing approaches in reducing makespan on benchmark instances.

Learning to Optimize Job Shop Scheduling Under Structural Uncertainty

TL;DR

This work tackles the Job-Shop Scheduling Problem under structural uncertainty, where routing paths are determined by uncertain, scenario-driven factors. It introduces UP-AAC, an Asymmetric Actor-Critic framework that trains the Critic on deterministic hindsight states while the Actor operates on stochastic states, enabling low-variance, stable policy updates; paired with the Uncertainty Perception Model (UPM) that injects a global risk representation into decisions. Empirically, UP-AAC achieves state-of-the-art makespan performance and robustness (CVaR) across small, medium, and large instances, with ablations showing both AAC and UPM are essential for gains. The approach offers a practical pathway to robust, uncertainty-aware scheduling in complex manufacturing settings and suggests extensions to leverage historical data for continual improvement.

Abstract

The Job-Shop Scheduling Problem (JSSP), under various forms of manufacturing uncertainty, has recently attracted considerable research attention. Most existing studies focus on parameter uncertainty, such as variable processing times, and typically adopt the actor-critic framework. In this paper, we explore a different but prevalent form of uncertainty in JSSP: structural uncertainty. Structural uncertainty arises when a job may follow one of several routing paths, and the selection is determined not by policy, but by situational factors (e.g., the quality of intermediate products) that cannot be known in advance. Existing methods struggle to address this challenge due to incorrect credit assignment: a high-quality action may be unfairly penalized if it is followed by a time-consuming path. To address this problem, we propose a novel method named UP-AAC. In contrast to conventional actor-critic methods, UP-AAC employs an asymmetric architecture. While its actor receives a standard stochastic state, the critic is crucially provided with a deterministic state reconstructed in hindsight. This design allows the critic to learn a more accurate value function, which in turn provides a lower-variance policy gradient to the actor, leading to more stable learning. In addition, we design an attention-based Uncertainty Perception Model (UPM) to enhance the actor's scheduling decisions. Extensive experiments demonstrate that our method outperforms existing approaches in reducing makespan on benchmark instances.
Paper Structure (26 sections, 5 equations, 3 figures, 3 tables, 1 algorithm)

This paper contains 26 sections, 5 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Deterministic Scheduling Methods
  4. Uncertainty-aware Methods
  5. Preliminaries and Problem Formulation
  6. Classical Job-Shop Scheduling Problem
  7. JSSP with Structural Uncertainty
  8. Standard Actor-Critic Framework
  9. Methodology
  10. Markov Decision Process Formulation
  11. State ($\mathcal{S}$)
  12. Action ($\mathcal{A}$)
  13. State Transition ($P$)
  14. Reward ($R$)
  15. Asymmetric Actor-Critic
  16. ...and 11 more sections

Figures (3)

  • Figure 1: An example of the credit assignment problem. The same dispatched operation can lead to drastically different makespans due to random path realizations. This unfairly rewards or penalizes the actor's choice, obscuring the action's true quality and hindering the learning process.
  • Figure 2: Overview of UP-AAC
  • Figure 3: Hindsight Reconstruction