Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Balanced Splitting: A Framework for Achieving Zero-wait in the Multiserver-job Model

Jonatha Anselmi, Josu Doncel

TL;DR

This work introduces Balanced Splitting Framework (BSF) for nonpreemptive, size-oblivious scheduling in the multiserver-job model, leveraging static, class-aware partitions $(\mathcal{A}_i)$ and a helper set $(\mathcal{H})$ to reduce interference. The policy Balanced Splitting-$\pi$ routes arrivals to the appropriate partition or the helper set and reassigns completed class-$i$ jobs to the corresponding partition; a central insight is that the helper-probability $P_{\mathcal{H}}$ vanishes in subcritical and Halfin-Whitt regimes, yielding zero-wait asymptotics. The analysis connects to the classical M/GI/$s$/$s$ queue via Erlang's loss formula to establish stability and asymptotics, with supporting simulations on synthetic data and real HPC traces showing competitive delays against preemptive and size-aware policies. These results provide a scalable, practical mechanism for zero-wait performance in large-scale parallel systems.

Abstract

We present a new framework for designing nonpreemptive and job-size oblivious scheduling policies in the multiserver-job queueing model. The main requirement is to identify a static and balanced sub-partition of the server set and ensure that the servers in each set of that sub-partition can only handle jobs of a given class and in a first-come first-served order. A job class is determined by the number of servers to which it has exclusive access during its entire execution and the probability distribution of its service time. This approach aims to reduce delays by preventing small jobs from being blocked by larger ones that arrived first, and it is particularly beneficial when the job size variability intra resp. inter classes is small resp. large. In this setting, we propose a new scheduling policy, Balanced-Splitting. We provide a sufficient condition for the stability of Balanced-Splitting and show that the resulting queueing probability, i.e., the probability that an arriving job needs to wait for processing upon arrival, vanishes in both the subcritical (the load is kept fixed to a constant less than one) and critical (the load approaches one from below) many-server limiting regimes. Crucial to our analysis is a connection with the M/GI/s/s queue and Erlang's loss formula, which allows our analysis to rely on fundamental results from queueing theory. Numerical simulations show that the proposed policy performs better than several preemptive/nonpreemptive size-aware/oblivious policies in various practical scenarios. This is also confirmed by simulations running on real traces from High Performance Computing (HPC) workloads. The delays induced by Balanced-Splitting are also competitive with those induced by state-of-the-art policies such as First-Fit-SRPT and ServerFilling-SRPT, though our approach has the advantage of not requiring preemption, nor the knowledge of job sizes.

Balanced Splitting: A Framework for Achieving Zero-wait in the Multiserver-job Model

TL;DR

This work introduces Balanced Splitting Framework (BSF) for nonpreemptive, size-oblivious scheduling in the multiserver-job model, leveraging static, class-aware partitions and a helper set to reduce interference. The policy Balanced Splitting- routes arrivals to the appropriate partition or the helper set and reassigns completed class- jobs to the corresponding partition; a central insight is that the helper-probability vanishes in subcritical and Halfin-Whitt regimes, yielding zero-wait asymptotics. The analysis connects to the classical M/GI// queue via Erlang's loss formula to establish stability and asymptotics, with supporting simulations on synthetic data and real HPC traces showing competitive delays against preemptive and size-aware policies. These results provide a scalable, practical mechanism for zero-wait performance in large-scale parallel systems.

Abstract

We present a new framework for designing nonpreemptive and job-size oblivious scheduling policies in the multiserver-job queueing model. The main requirement is to identify a static and balanced sub-partition of the server set and ensure that the servers in each set of that sub-partition can only handle jobs of a given class and in a first-come first-served order. A job class is determined by the number of servers to which it has exclusive access during its entire execution and the probability distribution of its service time. This approach aims to reduce delays by preventing small jobs from being blocked by larger ones that arrived first, and it is particularly beneficial when the job size variability intra resp. inter classes is small resp. large. In this setting, we propose a new scheduling policy, Balanced-Splitting. We provide a sufficient condition for the stability of Balanced-Splitting and show that the resulting queueing probability, i.e., the probability that an arriving job needs to wait for processing upon arrival, vanishes in both the subcritical (the load is kept fixed to a constant less than one) and critical (the load approaches one from below) many-server limiting regimes. Crucial to our analysis is a connection with the M/GI/s/s queue and Erlang's loss formula, which allows our analysis to rely on fundamental results from queueing theory. Numerical simulations show that the proposed policy performs better than several preemptive/nonpreemptive size-aware/oblivious policies in various practical scenarios. This is also confirmed by simulations running on real traces from High Performance Computing (HPC) workloads. The delays induced by Balanced-Splitting are also competitive with those induced by state-of-the-art policies such as First-Fit-SRPT and ServerFilling-SRPT, though our approach has the advantage of not requiring preemption, nor the knowledge of job sizes.
Paper Structure (25 sections, 7 theorems, 24 equations, 3 figures, 2 tables)

This paper contains 25 sections, 7 theorems, 24 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Common scheduling policies and positioning
  3. Balanced Splitting
  4. Main contribution
  5. Scheduling policies review
  6. Queueing probability and the zero-wait property
  7. Framework
  8. Multiserver-job queueing model
  9. Partitioning the set of servers
  10. Balanced Splitting framework
  11. Problem statement
  12. Main results
  13. Preliminaries
  14. Stability
  15. Subcritically loaded many-server regime
  16. ...and 10 more sections

Key Result

Lemma 1

Suppose that the arrival rate $\lambda$ and the number of servers $s$ in an M/GI/$s$/$s$ queue grow to infinity while the load (or utilization) $\rho=\lambda d/s$ approaches one and $(1-\rho)\sqrt{s}\to\theta$ where $\theta$ is a fixed constant. Then, where $\Phi(x)$ and $\phi(x)$ denote the standard normal cumulative distribution function (CDF) and density, respectively.

Figures (3)

  • Figure 1: Simulated mean response time in the critically loaded limiting regime (Halfin--Whitt). Small resp. large jobs arrive with probability 0.95 resp. 0.05. Small jobs have server need and mean service time equal to $(f_k,1)$, where $f_k= \lfloor (k/32)^{2/3} \rfloor$. Large jobs have server need and mean service time equal to (2$f_k$,40), (4$f_k$,20) or (8$f_k$,10), with equal probability. Service times are exponentially distributed and all independent. Arrivals are Poisson. Each simulation run uses $10^6$ arrivals. The curves associated to ServerFilling-SRPT and First-Fit SRPT almost overlap.
  • Figure 2: Simulated mean response time in the heavy-traffic (a) and subcritical (b) regimes. Small resp. large jobs arrive with probability 0.95 resp. 0.05. Server needs and mean service times are as in Figure \ref{['fig:1']}. Arrivals are Poisson.
  • Figure 3: Simulated response times within the KIT FH2 and SDSC SP2 datasets by varying the network load and with respect to $k=512$ (left) and $k=1024$ (right) overall servers. The auxiliary policy $\pi$ used within BalanceSplitting (${\rm BS\hbox{-}\pi}$) is FCFS.

Theorems & Definitions (13)

  • Definition 1
  • Remark 1
  • Lemma 1
  • Proposition 1
  • Theorem 1
  • Theorem 2
  • Definition 2
  • Proposition 2
  • proof
  • Corollary 1
  • ...and 3 more