Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Adaptive Mixed-Criticality Scheduling with Deep Reinforcement Learning

Bruno Mendes, Pedro F. Souto, Pedro C. Diniz

TL;DR

This paper enhances Adaptive Mixed-Criticality with a deep reinforcement learning (DRL) approach based on a Deep-Q Network that is able to reduce budget overruns by at least up to 50%, even when the budget of each task is chosen based on sampling the distribution of its execution time.

Abstract

Adaptive Mixed-Criticality (AMC) is a fixed-priority preemptive scheduling algorithm for mixed-criticality hard real-time systems. It dominates many other scheduling algorithms for mixed-criticality systems, but does so at the cost of occasionally dropping jobs of less important/critical tasks, when low-priority jobs overrun their time budgets. In this paper we enhance AMC with a deep reinforcement learning (DRL) approach based on a Deep-Q Network. The DRL agent is trained off-line, and at run-time adjusts the low-criticality budgets of tasks to avoid budget overruns, while ensuring that no job misses its deadline if it does not overrun its budget. We have implemented and evaluated this approach by simulating realistic workloads from the automotive domain. The results show that the agent is able to reduce budget overruns by at least up to 50%, even when the budget of each task is chosen based on sampling the distribution of its execution time. To the best of our knowledge, this is the first use of DRL in AMC reported in the literature.

Enhancing Adaptive Mixed-Criticality Scheduling with Deep Reinforcement Learning

TL;DR

This paper enhances Adaptive Mixed-Criticality with a deep reinforcement learning (DRL) approach based on a Deep-Q Network that is able to reduce budget overruns by at least up to 50%, even when the budget of each task is chosen based on sampling the distribution of its execution time.

Abstract

Adaptive Mixed-Criticality (AMC) is a fixed-priority preemptive scheduling algorithm for mixed-criticality hard real-time systems. It dominates many other scheduling algorithms for mixed-criticality systems, but does so at the cost of occasionally dropping jobs of less important/critical tasks, when low-priority jobs overrun their time budgets. In this paper we enhance AMC with a deep reinforcement learning (DRL) approach based on a Deep-Q Network. The DRL agent is trained off-line, and at run-time adjusts the low-criticality budgets of tasks to avoid budget overruns, while ensuring that no job misses its deadline if it does not overrun its budget. We have implemented and evaluated this approach by simulating realistic workloads from the automotive domain. The results show that the agent is able to reduce budget overruns by at least up to 50%, even when the budget of each task is chosen based on sampling the distribution of its execution time. To the best of our knowledge, this is the first use of DRL in AMC reported in the literature.

Paper Structure

This paper contains 35 sections, 7 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Model changes
  4. Design-Time changes
  5. Run-time changes
  6. Background
  7. AMC Scheduling
  8. System model
  9. Scheduling algorithm
  10. Schedulability Analysis
  11. Priority assignment
  12. Reinforcement Learning
  13. Traditional learning methods
  14. Deep Q-Network
  15. Approach
  16. ...and 20 more sections

Figures (5)

  • Figure 1: Percentage of guaranteed schedulable sets depending on the number of total runnables.
  • Figure 2: Agent execution times on a Raspberry Pi 4 Model B.
  • Figure 3: Number of task sets for which the hyperparameter values that we varied performed best, across simulations of 10 task sets with 150 runnables each.
  • Figure 4: Improvement observed in mode changes with and without an agent across 100 task sets with 150 and 250 runnables, excluding outliers.
  • Figure 5: Improvement observed in job cancellations with and without an agent across 100 task sets with 150 and 250 runnables, excluding outliers.