ReLA: Representation Learning and Aggregation for Job Scheduling with Reinforcement Learning

TL;DR

ReLA tackles the challenging problem of Flexible Job Shop Scheduling (FJSP) by learning and aggregating multiple, diverse representations of scheduling entities (operations and machines) to guide reinforcement learning decisions. It combines intra-entity learning (self-attention and convolution) with inter-entity learning (cross-attention) across a two-scale architecture, and uses three parallel representation modules to form rich, pairwise context for each feasible operation–machine assignment. The policy leverages six actor networks (two scales × three modules) to score candidate pairs, while a critic estimates state value from pooled representations, trained with PPO to minimize makespan $C_{max}$. Experiments show that ReLA achieves strong scheduling performance across small, medium, and large instances, outperforming prior learning-based methods and approaching or beating OR-Tools in many settings, with substantial gap reductions particularly on large-scale problems, indicating strong potential for real-world deployment in smart manufacturing.

Abstract

Job scheduling is widely used in real-world manufacturing systems to assign ordered job operations to machines under various constraints. Existing solutions remain limited by long running time or insufficient schedule quality, especially when problem scale increases. In this paper, we propose ReLA, a reinforcement-learning (RL) scheduler built on structured representation learning and aggregation. ReLA first learns diverse representations from scheduling entities, including job operations and machines, using two intra-entity learning modules with self-attention and convolution and one inter-entity learning module with cross-attention. These modules are applied in a multi-scale architecture, and their outputs are aggregated to support RL decision-making. Across experiments on small, medium, and large job instances, ReLA achieves the best makespan in most tested settings over the latest solutions. On non-large instances, ReLA reduces the optimality gap of the SOTA baseline by 13.0%, while on large-scale instances it reduces the gap by 78.6%, with the average optimality gaps lowered to 7.3% and 2.1%, respectively. These results confirm that ReLA's learned representations and aggregation provide strong decision support for RL scheduling, and enable fast job completion and decision-making for real-world applications.

Figures (1)

• Figure 1: ReLA architecture for FJSP scheduling. Multiple representation learning modules extract intra- and inter-entity scheduling representations, which are aggregated to enrich RL decision-making beyond single-representation designs.