A Multi-Objective Framework for Optimizing GPU-Enabled VM Placement in Cloud Data Centers with Multi-Instance GPU Technology
Ahmad Siavashi, Mahmoud Momtazpour
TL;DR
The paper tackles MIG-induced fragmentation in cloud data centers by formulating an online multi-objective ILP for MIG-enabled VM placement and proposing GRMU, a multi-stage framework with dual-basket GPU pooling, intra-GPU defragmentation, and inter-GPU consolidation. Using real Alibaba GPU trace data, GRMU improves workload acceptance by 22%, reduces active hardware by 17%, and keeps migrations to a minimum (1% of accepted MIG-enabled VMs). The approach balances large-profile and small-profile workloads while mitigating fragmentation and enhancing overall GPU utilization. This work provides a practical, MIG-aware resource management solution with significant implications for scaling GPU-sharing in cloud environments.
Abstract
The extensive use of GPUs in cloud computing and the growing need for multitenancy have driven the development of innovative solutions for efficient GPU resource management. Multi-Instance GPU (MIG) technology from NVIDIA enables shared GPU usage in cloud data centers by providing isolated instances. However, MIG placement rules often lead to fragmentation and suboptimal resource utilization. In this work, we formally model the MIG-enabled VM placement as a multi-objective Integer Linear Programming (ILP) problem aimed at maximizing request acceptance, minimizing active hardware usage, and reducing migration overhead. Building upon this formulation, we propose GRMU, a multi-stage placement framework designed to address MIG placement challenges. GRMU performs intra-GPU migrations for defragmentation of a single GPU and inter-GPU migrations for consolidation and resource efficiency. It also employs a quota-based partitioning approach to allocate GPUs into two distinct baskets: one for large-profile workloads and another for smaller-profile workloads. Each basket has predefined capacity limits, ensuring fair resource distribution and preventing large-profile workloads from monopolizing resources. Evaluations on a real-world Alibaba GPU cluster trace reveal that GRMU improves acceptance rates by 22%, reduces active hardware by 17%, and incurs migration for only 1% of MIG-enabled VMs, demonstrating its effectiveness in minimizing fragmentation and improving resource utilization.