Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mean field optimal Core Allocation across Malleable jobs

Zhouzi Li, Mor Harchol-Balter, Benjamin Berg

TL;DR

This work addresses the core allocation problem for malleable jobs across a fixed core pool, aiming to minimize mean holding costs when jobs can run on arbitrary core counts with concave speedups. It introduces a rigorous mean-field analysis yielding two policies: FW-CAM, which uses effective load to achieve queueing-free operation in the limit, and WHAM, a Whittle-index-based policy that remains effective outside the mean-field regime. The authors prove mean-field optimality for FW-CAM and establish MF-optimality for WHAM under mild conditions, supported by simulations showing rapid convergence to the MF lower bound and superiority over traditional heuristics. The results provide both theoretical insight and practical policies for data centers with diverse malleable workloads, extending the CAM literature to general speedup functions, GI distributions, and multi-class settings. The work has significant implications for scalable, low-latency scheduling in large-scale systems where parallelism exhibits diminishing returns and heterogeneous job characteristics.

Abstract

Modern data centers and cloud computing clusters are increasingly running workloads composed of malleable jobs. A malleable job can be parallelized across any number of cores, yet the job typically exhibits diminishing marginal returns for each additional core on which it runs. This can be seen in the concavity of a job's speedup function, which describes the job's processing speed as a function of the number of cores on which it runs. Given the prevalence of malleable jobs, several theoretical works have posed the problem of how to allocate a fixed number of cores across a stream of arriving malleable jobs so as to minimize the mean response time across jobs. We refer to this as the Core Allocation to Malleable jobs (CAM) problem. We solve the CAM problem under a highly general setting, allowing for multiple job classes, each with an arbitrary concave speedup function and holding costs (weight). Furthermore, we allow for generally distributed inter-arrival times and job sizes. We analyze the CAM problem in the mean field asymptotic regime and derive two distinct mean field optimal policies, FW-CAM and WHAM. FW-CAM is interesting because it demonstrates a new intuition: in the mean field regime, job sizes are not relevant in finding an optimal policy. WHAM (Whittle Allocation for Malleable jobs) is interesting because it is asymptotically optimal and also serves as a good heuristic even outside of the asymptotic regime. Notably, none of the policies previously proposed in the literature are mean field optimal when jobs may follow different speedup functions.

Mean field optimal Core Allocation across Malleable jobs

TL;DR

This work addresses the core allocation problem for malleable jobs across a fixed core pool, aiming to minimize mean holding costs when jobs can run on arbitrary core counts with concave speedups. It introduces a rigorous mean-field analysis yielding two policies: FW-CAM, which uses effective load to achieve queueing-free operation in the limit, and WHAM, a Whittle-index-based policy that remains effective outside the mean-field regime. The authors prove mean-field optimality for FW-CAM and establish MF-optimality for WHAM under mild conditions, supported by simulations showing rapid convergence to the MF lower bound and superiority over traditional heuristics. The results provide both theoretical insight and practical policies for data centers with diverse malleable workloads, extending the CAM literature to general speedup functions, GI distributions, and multi-class settings. The work has significant implications for scalable, low-latency scheduling in large-scale systems where parallelism exhibits diminishing returns and heterogeneous job characteristics.

Abstract

Modern data centers and cloud computing clusters are increasingly running workloads composed of malleable jobs. A malleable job can be parallelized across any number of cores, yet the job typically exhibits diminishing marginal returns for each additional core on which it runs. This can be seen in the concavity of a job's speedup function, which describes the job's processing speed as a function of the number of cores on which it runs. Given the prevalence of malleable jobs, several theoretical works have posed the problem of how to allocate a fixed number of cores across a stream of arriving malleable jobs so as to minimize the mean response time across jobs. We refer to this as the Core Allocation to Malleable jobs (CAM) problem. We solve the CAM problem under a highly general setting, allowing for multiple job classes, each with an arbitrary concave speedup function and holding costs (weight). Furthermore, we allow for generally distributed inter-arrival times and job sizes. We analyze the CAM problem in the mean field asymptotic regime and derive two distinct mean field optimal policies, FW-CAM and WHAM. FW-CAM is interesting because it demonstrates a new intuition: in the mean field regime, job sizes are not relevant in finding an optimal policy. WHAM (Whittle Allocation for Malleable jobs) is interesting because it is asymptotically optimal and also serves as a good heuristic even outside of the asymptotic regime. Notably, none of the policies previously proposed in the literature are mean field optimal when jobs may follow different speedup functions.
Paper Structure (47 sections, 17 theorems, 24 equations, 2 figures)

This paper contains 47 sections, 17 theorems, 24 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Speedup functions and their effect on running time
  3. System Load vs. Effective Load
  4. Asymptotic Regimes
  5. Our Approaches and Impacts
  6. Our Contributions
  7. Prior Work
  8. Core Allocation to Malleable jobs: Our Problem Setting
  9. Job Classes
  10. Arrival process:
  11. Job size distribution:
  12. Speedup function:
  13. Performance Metrics
  14. Holding Cost:
  15. Goal:
  16. ...and 32 more sections

Key Result

theorem 1

The optimal policy for the Relaxed problem is: Whenever a class-$i$ job arrives in the system, $k_i$ cores are immediately allocated to it until it completes, where $k_i$ is the solution to the following convex optimization problem

Figures (2)

  • Figure 1: Examples of speedup functions.
  • Figure 2: Performance evaluation of core allocation policies. The workload consists of three classes with various speedup functions and phase-type job size distributions. See Appendix \ref{['app:sim']} for details. As the number of cores scales, the arrival rate of each class scales proportionally to keep the system load fixed at $\mathbf{\rho=0.25}$.

Theorems & Definitions (27)

  • definition 1: Informal, WHAM for minimizing the mean response time
  • theorem 1: Theorem 1 in li2024rentgpusbudget
  • definition 2
  • definition 3: FW-CAM
  • lemma 1
  • lemma 2
  • lemma 3: Corollary 3 from li2025simple
  • lemma 4: No queueing for each class
  • theorem 2
  • definition 4: Our Time-average multi-armed bandit formulation
  • ...and 17 more