Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

REACH: Reinforcement Learning for Adaptive Microservice Rescheduling in the Cloud-Edge Continuum

Xu Bai, Muhammed Tawfiqul Islam, Rajkumar Buyya, Adel N. Toosi

TL;DR

This work tackles latency-sensitive microservice applications in the cloud–edge continuum by introducing REACH, an RL-based rescheduling framework that incrementally updates only necessary microservice placements to minimize end-to-end latency. Using PPO in a custom CEEnv simulator, REACH learns policies that adapt to resource heterogeneity and network delays, then deploys via a Kubernetes plugin to a real testbed. Experiments show significant latency reductions and improved stability under node failures and traffic surges, with manageable online overhead and rapid training in simulation. The approach demonstrates practical applicability for real-world cloud–edge environments and lays groundwork for multi-objective extensions such as energy and bandwidth optimization.

Abstract

Cloud computing, despite its advantages in scalability, may not always fully satisfy the low-latency demands of emerging latency-sensitive pervasive applications. The cloud-edge continuum addresses this by integrating the responsiveness of edge resources with cloud scalability. Microservice Architecture (MSA) characterized by modular, loosely coupled services, aligns effectively with this continuum. However, the heterogeneous and dynamic computing resource poses significant challenges to the optimal placement of microservices. We propose REACH, a novel rescheduling algorithm that dynamically adapts microservice placement in real time using reinforcement learning to react to fluctuating resource availability, and performance variations across distributed infrastructures. Extensive experiments on a real-world testbed demonstrate that REACH reduces average end-to-end latency by 7.9%, 10%, and 8% across three benchmark MSA applications, while effectively mitigating latency fluctuations and spikes.

REACH: Reinforcement Learning for Adaptive Microservice Rescheduling in the Cloud-Edge Continuum

TL;DR

This work tackles latency-sensitive microservice applications in the cloud–edge continuum by introducing REACH, an RL-based rescheduling framework that incrementally updates only necessary microservice placements to minimize end-to-end latency. Using PPO in a custom CEEnv simulator, REACH learns policies that adapt to resource heterogeneity and network delays, then deploys via a Kubernetes plugin to a real testbed. Experiments show significant latency reductions and improved stability under node failures and traffic surges, with manageable online overhead and rapid training in simulation. The approach demonstrates practical applicability for real-world cloud–edge environments and lays groundwork for multi-objective extensions such as energy and bandwidth optimization.

Abstract

Cloud computing, despite its advantages in scalability, may not always fully satisfy the low-latency demands of emerging latency-sensitive pervasive applications. The cloud-edge continuum addresses this by integrating the responsiveness of edge resources with cloud scalability. Microservice Architecture (MSA) characterized by modular, loosely coupled services, aligns effectively with this continuum. However, the heterogeneous and dynamic computing resource poses significant challenges to the optimal placement of microservices. We propose REACH, a novel rescheduling algorithm that dynamically adapts microservice placement in real time using reinforcement learning to react to fluctuating resource availability, and performance variations across distributed infrastructures. Extensive experiments on a real-world testbed demonstrate that REACH reduces average end-to-end latency by 7.9%, 10%, and 8% across three benchmark MSA applications, while effectively mitigating latency fluctuations and spikes.

Paper Structure

This paper contains 20 sections, 11 equations, 11 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Microservice Application Scheduling in Cloud-Edge Continuum
  4. System Model
  5. Problem Formulation
  6. Reinforcement Learning (RL) Model
  7. Reinforcement Learning Deployment Pipeline
  8. Experimental Settings
  9. Testbed Setup
  10. Workload
  11. Metrics
  12. Baseline Algorithms
  13. Performance Evaluation
  14. Convergence of the RL Agents
  15. Scalability Analysis
  16. ...and 5 more sections

Figures (11)

  • Figure 1: Vehicular computing leveraging the cloud–edge continuum
  • Figure 2: System Architecture
  • Figure 3: Interaction Between RL Agent And Environment
  • Figure 4: Example of State Transitions in Proposed RL Model
  • Figure 5: Constructed benchmark MSA applications with varying service invocation patterns and orders.
  • ...and 6 more figures