Improving Nonpreemptive Multiserver Job Scheduling with Quickswap

TL;DR

This work proposes and analyzes a class of scheduling policies called MSF-Quick Swap (MSFQ) that performs well and shows that variants of the MSFQ policy can greatly outperform MSF and FCFS on real-world multiserver job workloads.

Abstract

Modern data center workloads are composed of multiserver jobs, computational jobs that require multiple servers in order to run. A data center server can run many multiserver jobs in parallel, as long as it has sufficient resources to meet their demands. However, multiserver jobs are generally stateful, meaning that job preemptions incur significant overhead from saving and reloading the state associated with running jobs. Hence, most systems try to avoid these costly job preemptions altogether. Given these constraints, a scheduling policy must determine what set of jobs to run in parallel at each moment in time to minimize the mean response time across a stream of arriving jobs. Unfortunately, simple non-preemptive policies such as FCFS may leave many servers idle, resulting in high mean response times or even system instability. Our goal is to design and analyze non-preemptive scheduling policies for multiserver jobs that maintain high system utilization to achieve low mean response time. One well-known non-preemptive policy, Most Servers First (MSF), prioritizes jobs with higher server requirements and achieves high resource utilization. However, MSF causes extreme variability in job waiting times, and can perform significantly worse than FCFS in practice. To address this, we propose and analyze a class of scheduling policies called MSF-Quick Swap (MSFQ) that performs well. MSFQ reduces the variability of job waiting times by periodically granting priority to other jobs in the system. We provide both stability results and an analysis of mean response time under MSFQ to prove that our policy dramatically outperforms MSF in the case where jobs request one or all the servers. In more complex cases, we evaluate MSFQ in simulation. We show that, with some additional optimization, variants of the MSFQ policy can greatly outperform MSF and FCFS on real-world multiserver job workloads.

Figures (8)

• Figure 1: Number of jobs in the system under MSF and MSFQ where there are $k$=32 servers, 90% of the job arrivals are 1-server jobs, mean job sizes are 1 for 1-server jobs and $k$-server jobs, and jobs arrive at a rate of 7.5 jobs/second.
• Figure 2: Impact of the threshold value, $\ell$, on mean response time of the MSFQ policy evaluated in \ref{['fig:rt_oneorN_32']}.
• Figure 3: Mean response time as a function of job arrival rate in a one-or-all MSJ system with $k=32$, $p_1=0.9$, and $\mu_1 = \mu_k = 1$. MSFQ beats all other non-preemptive policies in terms of both mean response time and weighted mean response time. In particular, MSFQ can be two orders of magnitude better than MSF and nMSR with respect to both metrics.
• Figure 4: Service phase durations for the MSFQ policy evaluated in \ref{['fig:rt_oneorN_32']}.
• Figure 5: Weighted mean response time as a function of job arrival rate in a 4-class MSJ system with $k=15$, $p_1=0.5$, $p_3=0.25$, $p_5=0.2$, and $p_{15}=0.05$.

Theorems & Definitions (32)

• Theorem 1: MSFQ Throughput-Optimality
• Theorem 2: MSFQ Summary Theorem
• Theorem 3
• proof
• Theorem 4
• proof
• Theorem 4: MSFQ Throughput-Optimality
• remark 1
• Lemma 1
• proof