Multiserver-job Response Time under Multilevel Scaling

TL;DR

This work analyzes multiserver-job systems under the load-focused multilevel scaling (LFMS) limit, concentrating on the 1-and-n model where jobs require either one server or all $n$ servers. It introduces the saturated-system framework and the relative-completions function $\Delta$ to connect throughput $\mu$ and the scaled mean queue length $E[Q(1-\rho)]=E[\Delta(Y_d)]$ across three regimes defined by the load fraction $p_n$. The authors derive explicit asymptotic expressions for $\mu$ and $E[\Delta(Y_d)]$ in the $n$-server dominated, balanced, and $1$-server dominated regimes, including a polynomial-scaling subcase $p_n=1/n^\alpha$, $\alpha>1$, and show a tipping point at balance with distinct delay scalings. Numerical experiments verify the asymptotics against exact fixed-$n$ formulas up to $n=10^8$, demonstrating tight multiplicative convergence in several regimes and highlighting the practical relevance of LFMS for large-scale MSJ systems.

Abstract

We study the multiserver-job setting in the load-focused multilevel scaling limit, where system load approaches capacity much faster than the growth of the number of servers $n$. We specifically consider the ``1 and $n$'' system, where each job requires either one server or all $n$ servers. Within the multilevel scaling limit, we examine three regimes: load dominated by $n$-server jobs, 1-server jobs, or balanced. In each regime, we characterize the asymptotic growth rate of the boundary of the stability region and the scaled mean queue length. We numerically verify our asymptotic results against exact formulae.

Figures (10)

• Figure 1: The distribution of number of CPUs requested in Google's Borg trace tirmazi_borg_2020. Number of CPUs is normalized to the size of the smallest request observed, not an absolute value. The peak of the distribution is around 500 normalized CPUs, and there is significant probability mass anywhere from $1$ to $10^5$ normalized CPUs.
• Figure 2: Exact versus asymptotic formulas in the $n$-server dominated load regime for $\mu$ and $\mathbb{E}[\Delta(Y_d)]$ as functions of $n$. Parametrization: $p_n=n^{-0.5}$ and $\mu_1=\mu_n=1$.
• Figure 3: Exact versus asymptotic formulas in the balanced load regime for $\mu$ and $\mathbb{E}[\Delta(Y_d)]$ as functions of $n$. Parametrization: $p_n=n^{-1}$ and $\mu_1=\mu_n=1$.
• Figure 4: Exact versus asymptotic formulas in the $1$-server dominated regime for $\mu$ and $\mathbb{E}[\Delta(Y_d)]$ as functions of $n$. Parametrization: $p_n=n^{-2}$ and $\mu_1=\mu_n=1$.
• Figure 5: Exact versus asymptotic formulas in the $n$-server dominated regime for $\mu$ and $\mathbb{E}[\Delta(Y_d)]$ as functions of $n$. Parametrization: $p_n=n^{-0.5}$ and $\mu_1=10, \mu_n=1$.

Theorems & Definitions (32)

• Theorem 4.1
• proof : Proof is given in \ref{['sec:regime-1']}, split into \ref{['lem:r1-mu']} and \ref{['thm:r1-delta']}
• Theorem 4.2
• proof : Proof is given in \ref{['sec:regime-2']}, split into \ref{['lem:r2-mu']} and \ref{['thm:r2-delta']}
• Theorem 4.3
• proof : Proof is given in \ref{['sec:r3-spec']}, split into \ref{['lem:r3-spec-mu']} and \ref{['thm:r3-spec-delta']}
• Lemma 5.1
• proof
• Lemma 5.2
• proof