Learning Interpretable Scheduling Algorithms for Data Processing Clusters
Zhibo Hu, Chen Wang, Helen, Paik, Yanfeng Shu, Liming Zhu
TL;DR
Scheduling DAG jobs in data-processing clusters is challenged by opaque DL policies and poor edge-case adaptability. The paper distills interpretable decision-tree schedulers from a DNN-based Decima model using trace-based learning, grouping candidate nodes to form an explainable predictor and pairing it with a fair executor-allocation strategy. The resulting Decision Tree Scheduler achieves high fidelity to the original DL decisions and often outperforms both the DL model and traditional heuristics, especially on larger or edge-case workloads, while remaining configurable and patchable. This approach offers a practical, transparent path to deployment in production clusters, enabling rapid adaptation to workload shifts through targeted tree tuning and multiple competing strategies. Overall, combining interpretable models with domain-informed heuristics yields strong performance gains and diagnostics for real-world data-parallel scheduling.
Abstract
Workloads in data processing clusters are often represented in the form of DAG (Directed Acyclic Graph) jobs. Scheduling DAG jobs is challenging. Simple heuristic scheduling algorithms are often adopted in practice in production data centres. There is much room for scheduling performance optimisation for cost saving. Recently, reinforcement learning approaches (like decima) have been attempted to optimise DAG job scheduling and demonstrate clear performance gain in comparison to traditional algorithms. However, reinforcement learning (RL) approaches face their own problems in real-world deployment. In particular, their black-box decision making processes and generalizability in unseen workloads may add a non-trivial burden to the cluster administrators. Moreover, adapting RL models on unseen workloads often requires significant amount of training data, which leaves edge cases run in a sub-optimal mode. To fill the gap, we propose a new method to distill a simple scheduling policy based on observations of the behaviours of a complex deep learning model. The simple model not only provides interpretability of scheduling decisions, but also adaptive to edge cases easily through tuning. We show that our method achieves high fidelity to the decisions made by deep learning models and outperforms these models when additional heuristics are taken into account.