Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Reinforcement Learning Based Backfilling Strategy for HPC Batch Jobs

Elliot Kolker-Hicks, Di Zhang, Dong Dai

TL;DR

This work reframes HPC backfilling as a reinforcement-learning problem and introduces RLBackfilling, which learns, via Proximal Policy Optimization, when and which queued jobs to backfill without relying solely on runtime prediction accuracy. It uses a kernel-based policy and a value network within an actor-critic framework, operating in a simulated environment to optimize the average bounded slowdown ($bsld$). Empirical results across real (e.g., SDSC-SP2, HPC2N) and synthetic (Lublin-1, Lublin-2) traces show RLBackfilling consistently outperforms EASY backfilling with both user-provided and actual runtimes, with improvements up to 59% and 30% respectively, and demonstrate its generality across unseen workloads. The findings indicate that learning-based backfilling can adapt to different base schedulers and workload patterns, offering practical gains for HPC throughput and utilization and opening avenues for broader future work in this area.

Abstract

High Performance Computing (HPC) systems are used across a wide range of disciplines for both large and complex computations. HPC systems often receive many thousands of computational tasks at a time, colloquially referred to as jobs. These jobs must then be scheduled as optimally as possible so they can be completed within a reasonable timeframe. HPC scheduling systems often employ a technique called backfilling, wherein low-priority jobs are scheduled earlier to use the available resources that are waiting for the pending high-priority jobs. To make it work, backfilling largely relies on job runtime to calculate the start time of the ready-to-schedule jobs and avoid delaying them. It is a common belief that better estimations of job runtime will lead to better backfilling and more effective scheduling. However, our experiments show a different conclusion: there is a missing trade-off between prediction accuracy and backfilling opportunities. To learn how to achieve the best trade-off, we believe reinforcement learning (RL) can be effectively leveraged. Reinforcement Learning relies on an agent which makes decisions from observing the environment, and gains rewards or punishments based on the quality of its decision-making. Based on this idea, we designed RLBackfilling, a reinforcement learning-based backfilling algorithm. We show how RLBackfilling can learn effective backfilling strategies via trial-and-error on existing job traces. Our evaluation results show up to 59% better scheduling performance (based on average bounded job slowdown) compared to EASY backfilling using user-provided job runtime and 30% better performance compared with EASY using the ideal predicted job runtime (the actual job runtime).

A Reinforcement Learning Based Backfilling Strategy for HPC Batch Jobs

TL;DR

This work reframes HPC backfilling as a reinforcement-learning problem and introduces RLBackfilling, which learns, via Proximal Policy Optimization, when and which queued jobs to backfill without relying solely on runtime prediction accuracy. It uses a kernel-based policy and a value network within an actor-critic framework, operating in a simulated environment to optimize the average bounded slowdown (). Empirical results across real (e.g., SDSC-SP2, HPC2N) and synthetic (Lublin-1, Lublin-2) traces show RLBackfilling consistently outperforms EASY backfilling with both user-provided and actual runtimes, with improvements up to 59% and 30% respectively, and demonstrate its generality across unseen workloads. The findings indicate that learning-based backfilling can adapt to different base schedulers and workload patterns, offering practical gains for HPC throughput and utilization and opening avenues for broader future work in this area.

Abstract

High Performance Computing (HPC) systems are used across a wide range of disciplines for both large and complex computations. HPC systems often receive many thousands of computational tasks at a time, colloquially referred to as jobs. These jobs must then be scheduled as optimally as possible so they can be completed within a reasonable timeframe. HPC scheduling systems often employ a technique called backfilling, wherein low-priority jobs are scheduled earlier to use the available resources that are waiting for the pending high-priority jobs. To make it work, backfilling largely relies on job runtime to calculate the start time of the ready-to-schedule jobs and avoid delaying them. It is a common belief that better estimations of job runtime will lead to better backfilling and more effective scheduling. However, our experiments show a different conclusion: there is a missing trade-off between prediction accuracy and backfilling opportunities. To learn how to achieve the best trade-off, we believe reinforcement learning (RL) can be effectively leveraged. Reinforcement Learning relies on an agent which makes decisions from observing the environment, and gains rewards or punishments based on the quality of its decision-making. Based on this idea, we designed RLBackfilling, a reinforcement learning-based backfilling algorithm. We show how RLBackfilling can learn effective backfilling strategies via trial-and-error on existing job traces. Our evaluation results show up to 59% better scheduling performance (based on average bounded job slowdown) compared to EASY backfilling using user-provided job runtime and 30% better performance compared with EASY using the ideal predicted job runtime (the actual job runtime).
Paper Structure (25 sections, 4 figures, 5 tables)

This paper contains 25 sections, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. HPC Scheduling and Backfilling
  4. Job Attributes
  5. HPC Job Scheduling
  6. HPC Job Backfilling
  7. Reinforcement Learning
  8. Reinforcement Learning and Deep Neural Network Approaches
  9. Backfilling Defined as a Reinforcement-Learning Problem
  10. Design and Implementation
  11. Overview
  12. Observation
  13. RL Agent
  14. Policy Network.
  15. Value Network.
  16. ...and 10 more sections

Figures (4)

  • Figure 1: bsld performance of different prediction accuracy on SDSC-SP2 using different scheduling algorithms.
  • Figure 2: An illustrative example of trade-off regarding runtime prediction accuracy.
  • Figure 3: Overall architecture of RLBackfilling.
  • Figure 4: The training curves of RLBackfilling on four job traces (SDSC-SP2, HPC2N, Lublin-1, Lublin-2). x-axis shows the training epoch; y-axis shows the $bsld$ results.