Learning Virtual Machine Scheduling in Cloud Computing through Language Agents
JieHao Wu, Ziwei Wang, Junjie Sheng, Wenhao Li, Xiangfeng Wang, Jun Luo
TL;DR
The paper tackles VM scheduling under Online Dynamic Multidimensional Bin Packing by treating it as a $SMDP$-Option problem and introducing MiCo, a hierarchical framework that uses LLM-driven heuristics to automatically discover and compose scheduling policies. It decomposes policy learning into Option Miner (scenario-specific option discovery) and Option Composer (master policy over options), enabling robust performance in large-scale, nonstationary cloud workloads. Empirical results on Huawei-East-1 and Azure data show MiCo achieving up to 96.9% of the offline upper bound with strong robustness and favorable comparisons to baselines like SchedRL and traditional heuristics, along with detailed ablations and interpretability analyses. The approach is also open-sourced, promoting broader exploration of LLM-based hyper-heuristics for complex, dynamic optimization problems in cloud resource management.
Abstract
In cloud services, virtual machine (VM) scheduling is a typical Online Dynamic Multidimensional Bin Packing (ODMBP) problem, characterized by large-scale complexity and fluctuating demands. Traditional optimization methods struggle to adapt to real-time changes, domain-expert-designed heuristic approaches suffer from rigid strategies, and existing learning-based methods often lack generalizability and interpretability. To address these limitations, this paper proposes a hierarchical language agent framework named MiCo, which provides a large language model (LLM)-driven heuristic design paradigm for solving ODMBP. Specifically, ODMBP is formulated as a Semi-Markov Decision Process with Options (SMDP-Option), enabling dynamic scheduling through a two-stage architecture, i.e., Option Miner and Option Composer. Option Miner utilizes LLMs to discover diverse and useful non-context-aware strategies by interacting with constructed environments. Option Composer employs LLMs to discover a composing strategy that integrates the non-context-aware strategies with the contextual ones. Extensive experiments on real-world enterprise datasets demonstrate that MiCo achieves a 96.9\% competitive ratio in large-scale scenarios involving more than 10,000 virtual machines. It maintains high performance even under nonstationary request flows and diverse configurations, thus validating its effectiveness in complex and large-scale cloud environments.