Metronome: Efficient Scheduling for Periodic Traffic Jobs with Network and Priority Awareness

TL;DR

Metronome tackles scheduling for periodic traffic in cloud-native networks by jointly optimizing bandwidth quantity and timing. It introduces a geometric abstraction to interleave communication phases using time-division multiplexing and a three-stage multi-objective optimization that prioritizes utilization, latency, and cushion between contending tasks, all implemented as a Kubernetes-integrated system with custom resources, a plugin scheduler, and a stop-and-wait controller. The approach is validated on a heterogeneous GPU cluster with 13 ML models, demonstrating reductions in job completion time (up to 19.50%) and improvements in average bandwidth utilization (up to 23.20%) compared to standard Kubernetes scheduling. Key contributions include the 2D bandwidth resource model, offline recalculation for complex schemes, and continuous monitoring to maintain interleaving amid drift, enabling near-ideal performance for high-priority workloads in multi-tenant environments. The work presents practical, scalable mechanisms for cloud-native networks to efficiently coordinate resources in distributed training tasks.

Abstract

With the rapid growth in computing power demand, cloud native networks have emerged as a promising solution to address the challenges of efficient resource coordination, particularly in coping with the dynamic fluctuations of network bandwidth in clusters. We propose Metronome, a network-aware and priority-aware scheduling mechanism for cloud native networks. This mechanism is designed to support jobs that exhibit periodic traffic patterns and dynamic bandwidth demands, particularly in the context of distributed training. Specifically, Metronome employs a time-division multiplexing approach that leverages job traffic characteristics to construct an elastic network resource allocation model, enabling efficient bandwidth sharing across multiple jobs. In addition, it incorporates a multi-objective optimization strategy, jointly considering latency and job priorities to achieve globally optimal as well as dynamic resource allocation. Finally, Metronome adapts to the dynamic environment by monitoring the cluster and performing reconfiguration operations. Extensive experiments with 13 common machine learning models demonstrate that Metronome can enhance cluster resource utilization while guaranteeing service performance. Compared with the existing Kubernetes scheduling mechanisms across multiple scenarios, Metronome reduces job completion time by up to 19.50% while improving average bandwidth utilization by up to 23.20%.

Figures (17)

• Figure 1: Comparison of link utilization across different approaches.
• Figure 2: Workflow of Metronome mechanism.
• Figure 3: Metronome architecture.
• Figure 4: Testbed environment: a K8s cluster with heterogeneous devices.
• Figure 5: Time series of training jobs and their iteration time.