Evaluating Malleable Job Scheduling in HPC Clusters using Real-World Workloads

TL;DR

The paper tackles inefficiencies from rigid job scheduling in HPC clusters by evaluating malleable scheduling using real workload traces from Theta, Eagle, and Cori via the ElastiSim simulator. It introduces four malleable strategies plus a KeepPref variant that preserves each job's preferred node count, transforming rigid jobs with a speedup model and testing across malleability levels from 0% to 100%. Across Haswell, KNL, Eagle, and Theta, malleability yields substantial improvements in turnaround time, makespan, wait time, and node utilization, with gains varying by workload and strategy; KeepPref often provides robust performance, especially on Theta. The findings demonstrate that even modest adoption (e.g., 20%) can meaningfully enhance resource efficiency and user experience, supporting the case for integrating malleable scheduling into HPC resource management while acknowledging limitations such as data movement modeling and GPU heterogeneity.

Abstract

Optimizing resource utilization in high-performance computing (HPC) clusters is essential for maximizing both system efficiency and user satisfaction. However, traditional rigid job scheduling often results in underutilized resources and increased job waiting times. This work evaluates the benefits of resource elasticity, where the job scheduler dynamically adjusts the resource allocation of malleable jobs at runtime. Using real workload traces from the Cori, Eagle, and Theta supercomputers, we simulate varying proportions (0-100%) of malleable jobs with the ElastiSim software. We evaluate five job scheduling strategies, including a novel one that maintains malleable jobs at their preferred resource allocation when possible. Results show that, compared to fully rigid workloads, malleable jobs yield significant improvements across all key metrics. Considering the best-performing scheduling strategy for each supercomputer, job turnaround times decrease by 37-67%, job makespan by 16-65%, job wait times by 73-99%, and node utilization improves by 5-52%. Although improvements vary, gains remain substantial even at 20% malleable jobs. This work highlights important correlations between workload characteristics (e.g., job runtimes and node requirements), malleability proportions, and scheduling strategies. These findings confirm the potential of malleability to address inefficiencies in current HPC practices and demonstrate that even limited adoption can provide substantial advantages, encouraging its integration into HPC resource management.

Figures (9)

• Figure 1: Haswell: Raw data (a) shows artifacts from shared-node jobs and daily splits (vertical blue lines). The red line shows the capacity of 2388.0 nodes. After cleaning and merging, the data looks realistic (b).
• Figure 2: KNL: Node utilization (blue line) with 100% rigid jobs. Red lines mark warm-up and final submission; metrics are computed in between. Green areas illustrate the number of jobs in the queue.
• Figure 3: Haswell: Node counts and runtimes---most jobs are small and short.
• Figure 4: Haswell: Node utilization with 100% rigid jobs. Red lines mark warm-up and final submission; metrics are computed in between.
• Figure 5: Distribution of job sizes and runtimes. KNL: Most jobs use 4 or fewer nodes and run under 1000.0 s. Eagle: Jobs are nearly all single-node and short-lived. Theta: More even distribution, with peaks at 1, 8, and 256 nodes.