Strongly Tail-Optimal Scheduling in the Light-Tailed M/G/1

TL;DR

This paper addresses minimizing the tail of the response time in an M/G/1 queue with light-tailed job sizes by introducing Boost, a family of scheduling policies that adjust the effective arrival times via a boost function. The authors derive a closed-form expression for the optimal tail constant C and prove that the γ-Boost policy is strongly tail-optimal in the full-information setting, with extensions to partial information and promising practical performance. A key insight is the reduction to a batch-optimization problem using WDSPT/WDSEPT concepts and a cheating M/G/1 as a lower-bound tool, enabling rigorous tail-analysis via a tagged-job framework. Simulations corroborate substantial tail improvements over FCFS, Nudge, and SRPT across distributions and information regimes, suggesting both theoretical significance and practical applicability for high-reliability queueing systems.

Abstract

We study the problem of scheduling jobs in a queueing system, specifically an M/G/1 with light-tailed job sizes, to asymptotically optimize the response time tail. This means scheduling to make $\mathbf{P}[T > t]$, the chance a job's response time exceeds $t$, decay as quickly as possible in the $t \to \infty$ limit. For some time, the best known policy was First-Come First-Served (FCFS), which has an asymptotically exponential tail: $\mathbf{P}[T > t] \sim C e^{-γt}$. FCFS achieves the optimal *decay rate* $γ$, but its *tail constant* $C$ is suboptimal. Only recently have policies that improve upon FCFS's tail constant been discovered. But it is unknown what the optimal tail constant is, let alone what policy might achieve it. In this paper, we derive a closed-form expression for the optimal tail constant $C$, and we introduce *$γ$-Boost*, a new policy that achieves this optimal tail constant. Roughly speaking, $γ$-Boost operates similarly to FCFS, but it pretends that small jobs arrive earlier than their true arrival times. This significantly reduces the response time of small jobs without unduly delaying large jobs, improving upon FCFS's tail constant by up to 50% with only moderate job size variability, with even larger improvements for higher variability. While these results are for systems with full job size information, we also introduce and analyze a version of $γ$-Boost that works in settings with partial job size information, showing it too achieves significant gains over FCFS. Finally, we show via simulation that $γ$-Boost has excellent practical performance.

Figures (8)

• Figure 1: Comparison between how Nudge and Boost each handle a long job X arriving before a short job Y. Suppose that Y arrives before X enters service. Nudge decides the order to serve the jobs based only on the arrival order, as shown in (a). In contrast, Boost uses not just the arrival order but also the respective arrival times, as shown in (b) and (c). Notation: job $i$'s arrival time is $a_i$, its size is $s_i$, and its boost is $b(s_i)$.
• Figure 2: Empirical performance (higher is better) of , specifically the strongly tail-optimal γ, (a) on several job size distributions, and (b) compared to two other policies, Nudge (and the K and M variants with optimal parameter $K$) and SRPT. The plots show tail improvement ratio$1 - \P{T_\pi > t}/\P{T_{\textnormal{FCFS}} > t}$ as a function of $t$. Dotted lines indicate the asymptotic tail improvement ratio $1 - C_\pi/C_{\textnormal{FCFS}}$. The load is $\rho = 0.8$, and the mean job size is $\E{S} = 1$. See \ref{['sec:simulation']} for additional details on the job size distributions and other simulation parameters.
• Figure 3: Empirical TIR of γ over FCFS for several job size distributions $S$, each with mean $\E{S} = 1$, at loads $\rho = 0.5, 0.95$. See \ref{['fig:intro:performance:distributions']} for $\rho = 0.8$. We use the same distributions as grosof_nudge_2021, which are: Uniform($0$,$2$), Exponential, Hyperexponential with branches drawn from Exp($2$) and Exp($1/3$) and first branch probability $0.8$, and BoundedLomax with shape parameter $\alpha = 2$ and upper bound $4$. The asymptotic TIR is computed with \ref{['thm:general_transform']} and plotted as a same color dotted line for each distribution. Simulations run with 50 million arrivals.
• Figure 4: Comparison of empirical TIR of γ against FCFS and Nudge, for two job size distributions, each with mean $1$, and with the same settings as in grosof_nudge_2021. The left is Hyperexponential with branches drawn from Exp($2$) and Exp($1/3$), with first branch probability $0.8$ and the right is Uniform($0$,$2$). See \ref{['fig:intro:performance:policies']} for exponential. In each plot, the dotted horizontal line represents γ's asymptotic TIR for the respective distribution. For Hyperexponential, we set $K$ to the optimal value of $8$ for Nudge-K/M, with type-$1$ and type-$2$ jobs set to jobs coming from the Exp($2$) and Exp($1/3$) branches respectively. (Nudge-K does perform slightly better than Nudge in this case, though this is barely visible on the plot.) For Uniform, we use the same small-large split as Nudge, where type-$1$ jobs are small (smaller than the mean of the distribution) and type-$2$ jobs are large, and set $K$ to the optimal value of $3$ for Nudge-K/M. Simulations run with 50 million arrivals.
• Figure 5: A comparison of γ and SRPT at high $\mathrm{CoV}$. In both plots, the distributions considered are Hyperexponential, mean $1$ distributions. For $\mathrm{CoV}^2 = 8.5$, we choose parameters Exp(4), Exp(1/6), and a first-branch probability of $p = 20/23$. For $\mathrm{CoV}^2 = 20.25$ we choose parameters Exp(8), Exp(1/12), and a first-branch probability of $p = 88/95$. Load is $\rho = 0.8$. Dashed vertical lines mark the $t_{0.99}$ response times of the two policies, and dash-dotted vertical lines mark the $t_{0.999}$ response time. The dotted horizontal line represents the theoretical asymptotic TIR for γ. Observe how SRPT has lower $t_{0.99}$ response time but higher $t_{0.999}$ response time than γ. Simulations run with 50 million arrivals.

Theorems & Definitions (21)

• Definition 1
• Definition 2
• Definition 3
• Theorem 4
• Definition 5
• Lemma 6
• Remark 7
• Lemma 8
• Definition 9
• Definition 10