Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reinforcement Learning-Based Dynamic Management of Structured Parallel Farm Skeletons on Serverless Platforms

Lanpei Li, Massimo Coppola, Malio Li, Valerio Besozzi, Jack Bell, Vincenzo Lomonaco

TL;DR

This work tackles dynamic management of structured parallel skeletons on serverless platforms by implementing a Farm skeleton on OpenFaaS and framing worker-pool autoscaling as a QoS-aware control problem with deadline-based objectives. It introduces a Gymnasium-compatible environment and compares reactive baselines with two RL agents (SARSA and Double DQN) trained to optimize QoS, backlog, and scaling stability. Results show RL-based policies achieve higher QoS and more stable scaling than reactive approaches, with SARSA providing the strongest guarantees and DQN offering a favorable cost-stability trade-off. The framework demonstrates the potential of AI-driven autoscaling to compensate for platform-induced delays and non-idealities, enabling HPC-like performance guarantees in serverless settings and guiding future extensions to multi-skeleton, multi-tenant deployments and richer cost models.

Abstract

We present a framework for dynamic management of structured parallel processing skeletons on serverless platforms. Our goal is to bring HPC-like performance and resilience to serverless and continuum environments while preserving the programmability benefits of skeletons. As a first step, we focus on the well known Farm pattern and its implementation on the open-source OpenFaaS platform, treating autoscaling of the worker pool as a QoS-aware resource management problem. The framework couples a reusable farm template with a Gymnasium-based monitoring and control layer that exposes queue, timing, and QoS metrics to both reactive and learning-based controllers. We investigate the effectiveness of AI-driven dynamic scaling for managing the farm's degree of parallelism via the scalability of serverless functions on OpenFaaS. In particular, we discuss the autoscaling model and its training, and evaluate two reinforcement learning (RL) policies against a baseline of reactive management derived from a simple farm performance model. Our results show that AI-based management can better accommodate platform-specific limitations than purely model-based performance steering, improving QoS while maintaining efficient resource usage and stable scaling behaviour.

Reinforcement Learning-Based Dynamic Management of Structured Parallel Farm Skeletons on Serverless Platforms

TL;DR

This work tackles dynamic management of structured parallel skeletons on serverless platforms by implementing a Farm skeleton on OpenFaaS and framing worker-pool autoscaling as a QoS-aware control problem with deadline-based objectives. It introduces a Gymnasium-compatible environment and compares reactive baselines with two RL agents (SARSA and Double DQN) trained to optimize QoS, backlog, and scaling stability. Results show RL-based policies achieve higher QoS and more stable scaling than reactive approaches, with SARSA providing the strongest guarantees and DQN offering a favorable cost-stability trade-off. The framework demonstrates the potential of AI-driven autoscaling to compensate for platform-induced delays and non-idealities, enabling HPC-like performance guarantees in serverless settings and guiding future extensions to multi-skeleton, multi-tenant deployments and richer cost models.

Abstract

We present a framework for dynamic management of structured parallel processing skeletons on serverless platforms. Our goal is to bring HPC-like performance and resilience to serverless and continuum environments while preserving the programmability benefits of skeletons. As a first step, we focus on the well known Farm pattern and its implementation on the open-source OpenFaaS platform, treating autoscaling of the worker pool as a QoS-aware resource management problem. The framework couples a reusable farm template with a Gymnasium-based monitoring and control layer that exposes queue, timing, and QoS metrics to both reactive and learning-based controllers. We investigate the effectiveness of AI-driven dynamic scaling for managing the farm's degree of parallelism via the scalability of serverless functions on OpenFaaS. In particular, we discuss the autoscaling model and its training, and evaluate two reinforcement learning (RL) policies against a baseline of reactive management derived from a simple farm performance model. Our results show that AI-based management can better accommodate platform-specific limitations than purely model-based performance steering, improving QoS while maintaining efficient resource usage and stable scaling behaviour.
Paper Structure (16 sections, 4 equations, 6 figures, 5 tables)

This paper contains 16 sections, 4 equations, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Parallel Skeletons and the OpenFaaS Serverless Platform
  4. Problem formulation
  5. Farm Skeleton Implementation
  6. Customized Invocation and Scaling
  7. Function and Process Realization
  8. Workload and QoS Modeling
  9. Autoscaling Environment
  10. Implementation Considerations and Limitations of the Farm Skeleton on OpenFaaS
  11. Farm Dynamic Management Policies
  12. AI Approach: SARSA and DQN
  13. Experimental settings
  14. Task Arrival Patterns and Sizes
  15. Evaluation Methodology
  16. ...and 1 more sections

Figures (6)

  • Figure 1: OpenFaaS-based farm skeleton template.
  • Figure 2: Scaling metrics for the farm skeleton: runtime, initialization overhead (left) and speedup (right) vs. worker count, under static workloads.
  • Figure 3: The reward function for SARSA and DQN
  • Figure 4: Configured mean of task arrival patterns across four workload phases.
  • Figure 5: Final QoS vs. mean workers (up) and number of scaling actions (down), over 10 runs (per-run points and overall average).
  • ...and 1 more figures