Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

"Two-Stagification": Job Dispatching in Large-Scale Clusters via a Two-Stage Architecture

Mert Yildiz, Alexey Rolich, Andrea Baiocchi

TL;DR

The paper tackles dispatching in FCFS server clusters and introduces a two-stage architecture that offloads large jobs to a second stage using a service-time threshold $\theta$, while short jobs complete in the first stage. Through comparisons of simple (RR) and more informed policies (JIQ, LWL, CARD) on both synthetic Weibull workloads and Google traces, the study shows that two-stage offloading substantially improves mean response time and can approach the performance of state-aware methods as the degree of parallelism $n$ grows, especially under heavy-tailed workloads. CARD remains the strongest single-stage policy, but two-stage variants of RR, JIQ, and LWL frequently achieve near-CARD MRT with reduced coordination. These results suggest that architectural design choices can yield large performance gains in large-scale data centers, offering practical, scalable alternatives to deploying more complex dispatching rules.

Abstract

A continuing effort is devoted to devising effective dispatching policies for clusters of First Come First Served servers. Although the optimal solution for dispatchers aware of both job size and server state remains elusive, lower bounds and strong heuristics are known. In this paper, we introduce a two-stage cluster architecture that applies classical Round Robin, Join Idle Queue, and Least Work Left dispatching schemes, coupled with an optimized service-time threshold to separate large jobs from shorter ones. Using both synthetic (Weibull) workloads and real Google data center traces, we demonstrate that our two-stage approach greatly improves upon the corresponding single-stage policies and closely approaches the performance of advanced size- and state-aware methods. Our results highlight that careful architectural design-rather than increased complexity at the dispatcher-can yield significantly better mean response times in large-scale computing environments.

"Two-Stagification": Job Dispatching in Large-Scale Clusters via a Two-Stage Architecture

TL;DR

The paper tackles dispatching in FCFS server clusters and introduces a two-stage architecture that offloads large jobs to a second stage using a service-time threshold , while short jobs complete in the first stage. Through comparisons of simple (RR) and more informed policies (JIQ, LWL, CARD) on both synthetic Weibull workloads and Google traces, the study shows that two-stage offloading substantially improves mean response time and can approach the performance of state-aware methods as the degree of parallelism grows, especially under heavy-tailed workloads. CARD remains the strongest single-stage policy, but two-stage variants of RR, JIQ, and LWL frequently achieve near-CARD MRT with reduced coordination. These results suggest that architectural design choices can yield large performance gains in large-scale data centers, offering practical, scalable alternatives to deploying more complex dispatching rules.

Abstract

A continuing effort is devoted to devising effective dispatching policies for clusters of First Come First Served servers. Although the optimal solution for dispatchers aware of both job size and server state remains elusive, lower bounds and strong heuristics are known. In this paper, we introduce a two-stage cluster architecture that applies classical Round Robin, Join Idle Queue, and Least Work Left dispatching schemes, coupled with an optimized service-time threshold to separate large jobs from shorter ones. Using both synthetic (Weibull) workloads and real Google data center traces, we demonstrate that our two-stage approach greatly improves upon the corresponding single-stage policies and closely approaches the performance of advanced size- and state-aware methods. Our results highlight that careful architectural design-rather than increased complexity at the dispatcher-can yield significantly better mean response times in large-scale computing environments.
Paper Structure (14 sections, 4 equations, 6 figures)

This paper contains 14 sections, 4 equations, 6 figures.

Table of Contents

  1. Introduction
  2. System Model and Dispatching Policies
  3. Artificial workload
  4. Experimental workload dataset
  5. Dispatching policy
  6. Round Robin
  7. Join Idle Queue
  8. Least Work Left
  9. Multi-band flexible CARD
  10. Two-stage dispatching
  11. Numerical Results
  12. Performance under Weibull-Distributed Workload
  13. Performance under Real Google Workload
  14. Conclusions

Figures (6)

  • Figure 1: MRT as a function of the number of servers. Note:COV = 1.
  • Figure 2: MRT as a function of the number of servers. Note:COV = 10.
  • Figure 3: MRT as a function of the system load. Note:COV = 10 and n = 10.
  • Figure 4: MRT as a function of the number of servers. Note:COV = 10 and $\rho = 0.8$.
  • Figure 5: MRT as a function of the system load. Google data has been used including the task-level data structure. Note:$n = 10$
  • ...and 1 more figures