Learning to Schedule Online Tasks with Bandit Feedback

TL;DR

This work tackles online task scheduling under bandit feedback when rewards, costs, and task-arrival distributions are all unknown. It devises DOL-RM, a framework that blends double-optimistic learning for unknown rewards and costs with a Robbins-Monro style update to implicitly learn the arrival distribution and balance reward against cost. The authors prove a sub-linear regret bound of $O(T^{3/4})$ and a convergence gap of $O(T^{-1/4})$, established via a decomposition into learning and RM components and Lyapunov drift analysis. Empirical results on synthetic simulations and real-world ML task scheduling show DOL-RM achieving the best cumulative reward-to-cost ratio compared with state-of-the-art baselines, highlighting its practical impact for uncertain cloud and crowdsourcing environments.

Abstract

Online task scheduling serves an integral role for task-intensive applications in cloud computing and crowdsourcing. Optimal scheduling can enhance system performance, typically measured by the reward-to-cost ratio, under some task arrival distribution. On one hand, both reward and cost are dependent on task context (e.g., evaluation metric) and remain black-box in practice. These render reward and cost hard to model thus unknown before decision making. On the other hand, task arrival behaviors remain sensitive to factors like unpredictable system fluctuation whereby a prior estimation or the conventional assumption of arrival distribution (e.g., Poisson) may fail. This implies another practical yet often neglected challenge, i.e., uncertain task arrival distribution. Towards effective scheduling under a stationary environment with various uncertainties, we propose a double-optimistic learning based Robbins-Monro (DOL-RM) algorithm. Specifically, DOL-RM integrates a learning module that incorporates optimistic estimation for reward-to-cost ratio and a decision module that utilizes the Robbins-Monro method to implicitly learn task arrival distribution while making scheduling decisions. Theoretically, DOL-RM achieves convergence gap and no regret learning with a sub-linear regret of $O(T^{3/4})$, which is the first result for online task scheduling under uncertain task arrival distribution and unknown reward and cost. Our numerical results in a synthetic experiment and a real-world application demonstrate the effectiveness of DOL-RM in achieving the best cumulative reward-to-cost ratio compared with other state-of-the-art baselines.

Figures (5)

• Figure 1: Rewards and Costs for $(S,a)$ in a Toy System.
• Figure 2: Online Task Scheduling in Cloud Computing (above) and Crowdsourcing (bottom).
• Figure 3: Illustration of DOL-RM Algorithm.
• Figure 4: Synthetic Experiment: Performance Comparison in a Two-Task-Type Case and Seven-Task-Type Case.
• Figure 5: Real-World Experiment.

Theorems & Definitions (7)

• theorem 1
• lemma 1
• lemma 2
• lemma 3
• lemma 4
• lemma 5
• lemma 6: Lemma $3$ in Nee_21