Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Resource Efficiency with Fairness in Heterogeneous GPU Clusters

Zizhao Mo, Huanle Xu, Wing Cheong Lau

TL;DR

The paper tackles the challenge of maximizing training throughput while enforcing fairness in heterogeneous GPU clusters. It introduces OEF, a global optimization framework that unifies resource efficiency and multiple fairness properties across cooperative and non-cooperative settings, using profiling, convex optimization, and adjacency-aware placement. The approach provides theoretical guarantees including strategy-proofness and envy-freeness, while delivering substantial practical gains, such as up to 32% throughput improvements and 17–19% reductions in job completion time in real and large-scale traces, with modest overhead. This work offers a principled, scalable solution for multi-tenant DL workloads on diverse GPU ecosystems, with clear implications for cloud providers deploying heterogeneous accelerators.

Abstract

Ensuring the highest training throughput to maximize resource efficiency, while maintaining fairness among users, is critical for deep learning (DL) training in heterogeneous GPU clusters. However, current DL schedulers provide only limited fairness properties and suboptimal training throughput, impeding tenants from effectively leveraging heterogeneous resources. The underlying design challenge stems from inherent conflicts between efficiency and fairness properties. In this paper, we introduce OEF, a new resource allocation framework specifically developed for achieving optimal resource efficiency and ensuring diverse fairness properties in heterogeneous GPU clusters. By integrating resource efficiency and fairness within a global optimization framework, OEF is capable of providing users with maximized overall efficiency, as well as various guarantees of fairness, in both cooperative and non-cooperative environments. We have implemented OEF in a cluster resource manager and conducted large-scale experiments, showing that OEF can improve the overall training throughput by up to 32% while improving fairness compared to state-of-the-art heterogeneity-aware schedulers.

Optimal Resource Efficiency with Fairness in Heterogeneous GPU Clusters

TL;DR

The paper tackles the challenge of maximizing training throughput while enforcing fairness in heterogeneous GPU clusters. It introduces OEF, a global optimization framework that unifies resource efficiency and multiple fairness properties across cooperative and non-cooperative settings, using profiling, convex optimization, and adjacency-aware placement. The approach provides theoretical guarantees including strategy-proofness and envy-freeness, while delivering substantial practical gains, such as up to 32% throughput improvements and 17–19% reductions in job completion time in real and large-scale traces, with modest overhead. This work offers a principled, scalable solution for multi-tenant DL workloads on diverse GPU ecosystems, with clear implications for cloud providers deploying heterogeneous accelerators.

Abstract

Ensuring the highest training throughput to maximize resource efficiency, while maintaining fairness among users, is critical for deep learning (DL) training in heterogeneous GPU clusters. However, current DL schedulers provide only limited fairness properties and suboptimal training throughput, impeding tenants from effectively leveraging heterogeneous resources. The underlying design challenge stems from inherent conflicts between efficiency and fairness properties. In this paper, we introduce OEF, a new resource allocation framework specifically developed for achieving optimal resource efficiency and ensuring diverse fairness properties in heterogeneous GPU clusters. By integrating resource efficiency and fairness within a global optimization framework, OEF is capable of providing users with maximized overall efficiency, as well as various guarantees of fairness, in both cooperative and non-cooperative environments. We have implemented OEF in a cluster resource manager and conducted large-scale experiments, showing that OEF can improve the overall training throughput by up to 32% while improving fairness compared to state-of-the-art heterogeneity-aware schedulers.
Paper Structure (45 sections, 7 theorems, 15 equations, 17 figures, 1 table)

This paper contains 45 sections, 7 theorems, 15 equations, 17 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Multi-tenant GPU Clusters
  4. Diverse Speedup under GPU Heterogeneity
  5. Fair Allocation in Heterogeneous Environment
  6. Fairness requirements
  7. Optimal resource efficiency
  8. Why need new fair schedulers?
  9. Limitation of Existing Heterogeneous Schedulers
  10. OEF Overview
  11. Design Challenges
  12. Conflicts between efficiency and fairness
  13. Conflicts between various fairness properties
  14. Architecture & Workflow
  15. OEF Design
  16. ...and 30 more sections

Key Result

lemma 1

To achieve optimal efficiency, in $\vec{x_1}$, which represents the allocation for the user with the slowest speedup, all non-zero elements are positioned at the leftmost positions and have a value of $m_j$, except for the right-most element.

Figures (17)

  • Figure 1: Diverse speedup for users with different DL training jobs
  • Figure 2: Diverse speedup for users under different fair schedulers
  • Figure 4: Before lying
  • Figure 5: After lying
  • Figure 7: The system architecture of OEF.
  • ...and 12 more figures

Theorems & Definitions (7)

  • lemma 1
  • theorem 1
  • theorem 2
  • theorem 3
  • theorem 4
  • theorem 5
  • theorem 6