Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Energy-Efficient Computation with DVFS using Deep Reinforcement Learning for Multi-Task Systems in Edge Computing

Xinyi Li, Ti Zhou, Haoyu Wang, Man Lin

TL;DR

This work tackles energy-efficient computation for soft real-time, multitask edge systems by integrating reinforcement-learning–driven DVFS directly into the Linux kernel. It introduces two multitask state encoders (TEGM with GRU+MLP and TEM with MLP) and trains a Double Deep Q-Network in userspace to select frequencies, while the kernel performs fast inference and data exchange with the trainer. A reward function combines low-frequency energy, utilization, and a deadline-exceedance penalty, enabling policies that keep tasks within their deadlines while reducing power consumption. Evaluations on an Nvidia Jetson Nano 2GB across fixed and random multitask workloads show 3–10% power savings over built-in governors, with up to 15% in random task sets, and acceptable deadline miss rates, validating the method’s practicality for small embedded devices.

Abstract

Finding an optimal energy-efficient policy that is adaptable to underlying edge devices while meeting deadlines for tasks has always been challenging. This research studies generalized systems with multi-task, multi-deadline scenarios with reinforcement learning-based DVFS for energy saving for periodic soft real-time applications on edge devices. This work addresses the limitation of previous work that models a periodic system as a single task and single-deadline scenario, which is too simplified to cope with complex situations. The method encodes time series data in the Linux kernel into information that is easy to interpret for reinforcement learning, allowing the system to generate DVFS policies to adapt system patterns based on the general workload. For encoding, we present two different methods for comparison. Both methods use only one performance counter: system utilization, and the kernel only needs minimal information from the userspace. Our method is implemented on Jetson Nano Board (2GB) and is tested with three fixed multitask workloads, which are three, five, and eight tasks in the workload, respectively. For randomness and generalization, we also designed a random workload generator to build different multitask workloads to test. Based on the test results, our method could save 3%-10% power compared to Linux built-in governors.

Energy-Efficient Computation with DVFS using Deep Reinforcement Learning for Multi-Task Systems in Edge Computing

TL;DR

This work tackles energy-efficient computation for soft real-time, multitask edge systems by integrating reinforcement-learning–driven DVFS directly into the Linux kernel. It introduces two multitask state encoders (TEGM with GRU+MLP and TEM with MLP) and trains a Double Deep Q-Network in userspace to select frequencies, while the kernel performs fast inference and data exchange with the trainer. A reward function combines low-frequency energy, utilization, and a deadline-exceedance penalty, enabling policies that keep tasks within their deadlines while reducing power consumption. Evaluations on an Nvidia Jetson Nano 2GB across fixed and random multitask workloads show 3–10% power savings over built-in governors, with up to 15% in random task sets, and acceptable deadline miss rates, validating the method’s practicality for small embedded devices.

Abstract

Finding an optimal energy-efficient policy that is adaptable to underlying edge devices while meeting deadlines for tasks has always been challenging. This research studies generalized systems with multi-task, multi-deadline scenarios with reinforcement learning-based DVFS for energy saving for periodic soft real-time applications on edge devices. This work addresses the limitation of previous work that models a periodic system as a single task and single-deadline scenario, which is too simplified to cope with complex situations. The method encodes time series data in the Linux kernel into information that is easy to interpret for reinforcement learning, allowing the system to generate DVFS policies to adapt system patterns based on the general workload. For encoding, we present two different methods for comparison. Both methods use only one performance counter: system utilization, and the kernel only needs minimal information from the userspace. Our method is implemented on Jetson Nano Board (2GB) and is tested with three fixed multitask workloads, which are three, five, and eight tasks in the workload, respectively. For randomness and generalization, we also designed a random workload generator to build different multitask workloads to test. Based on the test results, our method could save 3%-10% power compared to Linux built-in governors.
Paper Structure (19 sections, 15 figures, 7 tables, 4 algorithms)

This paper contains 19 sections, 15 figures, 7 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Temperal Encoder
  4. Temporal Encoder (TEGM) for Network using GRU and MLP
  5. Temporal Encoder (TEM) for Network only using MLP
  6. Inference Module in Kernel
  7. Training Module in Userspace
  8. Experiment
  9. Experiment Setup
  10. Reward Curve
  11. Deadline Awareness
  12. Learned Policy
  13. Energy Saving
  14. Inference Time Overhead
  15. Deadline missing
  16. ...and 4 more sections

Figures (15)

  • Figure 1: Deadline Missing Example: A Multi-Task set with Multi-Deadline on Multi-Core
  • Figure 2: An overview of our proposed method.
  • Figure 3: Interaction between userspace and kernel.
  • Figure 4: TEGM example.
  • Figure 5: TEM example.
  • ...and 10 more figures