Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Reinforcement Learning for Delay-Optimized Task Offloading in Vehicular Fog Computing

Mohammad Parsa Toopchinezhad, Mahmood Ahmadi

TL;DR

The paper tackles delay-sensitive task offloading in Vehicular Fog Computing (VFC) by formulating the problem as a queuing-network with a centralized RSU. It adopts a Deep Reinforcement Learning approach (PPO) within a grid-based, realistic mobility simulator and releases an open-source environment for VFC offloading. The proposed method outperforms Random, Cloud-only, and Greedy baselines, achieving lower average delays and reduced queue congestion, with stronger gains as fleet size grows. This work demonstrates the scalability and practicality of fog-based offloading for large autonomous vehicle fleets, offering a realistic, reusable framework for future research.

Abstract

The imminent rise of autonomous vehicles (AVs) is revolutionizing the future of transport. The Vehicular Fog Computing (VFC) paradigm has emerged to alleviate the load of compute-intensive and delay-sensitive AV programs via task offloading to nearby vehicles. Effective VFC requires an intelligent and dynamic offloading algorithm. As a result, this paper adapts Deep Reinforcement Learning (DRL) for VFC offloading. First, a simulation environment utilizing realistic hardware and task specifications, in addition to a novel vehicular movement model based on grid-planned cities, is created. Afterward, a DRL-based algorithm is trained and tested on the environment with the goal of minimizing global task delay. The DRL model displays impressive results, outperforming other greedy and conventional methods. The findings further demonstrate the effectiveness of the DRL model in minimizing queue congestion, especially when compared to traditional cloud computing methods that struggle to handle the demands of a large fleet of vehicles. This is corroborated by queuing theory, highlighting the self-scalability of the VFC-based DRL approach.

Deep Reinforcement Learning for Delay-Optimized Task Offloading in Vehicular Fog Computing

TL;DR

The paper tackles delay-sensitive task offloading in Vehicular Fog Computing (VFC) by formulating the problem as a queuing-network with a centralized RSU. It adopts a Deep Reinforcement Learning approach (PPO) within a grid-based, realistic mobility simulator and releases an open-source environment for VFC offloading. The proposed method outperforms Random, Cloud-only, and Greedy baselines, achieving lower average delays and reduced queue congestion, with stronger gains as fleet size grows. This work demonstrates the scalability and practicality of fog-based offloading for large autonomous vehicle fleets, offering a realistic, reusable framework for future research.

Abstract

The imminent rise of autonomous vehicles (AVs) is revolutionizing the future of transport. The Vehicular Fog Computing (VFC) paradigm has emerged to alleviate the load of compute-intensive and delay-sensitive AV programs via task offloading to nearby vehicles. Effective VFC requires an intelligent and dynamic offloading algorithm. As a result, this paper adapts Deep Reinforcement Learning (DRL) for VFC offloading. First, a simulation environment utilizing realistic hardware and task specifications, in addition to a novel vehicular movement model based on grid-planned cities, is created. Afterward, a DRL-based algorithm is trained and tested on the environment with the goal of minimizing global task delay. The DRL model displays impressive results, outperforming other greedy and conventional methods. The findings further demonstrate the effectiveness of the DRL model in minimizing queue congestion, especially when compared to traditional cloud computing methods that struggle to handle the demands of a large fleet of vehicles. This is corroborated by queuing theory, highlighting the self-scalability of the VFC-based DRL approach.
Paper Structure (17 sections, 17 equations, 6 figures, 4 tables)

This paper contains 17 sections, 17 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Task Offloading in VFC
  4. RL-Based Task Offloading in VFC
  5. System Model
  6. Vehicle Model
  7. Task Model
  8. Delay Model
  9. Proposed Method
  10. State Space
  11. Action Space
  12. Reward Function
  13. Performance Evaluation
  14. Simulation Setup
  15. Delay Evaluation
  16. ...and 2 more sections

Figures (6)

  • Figure 1: VFC architecture; showcasing the three main layers along with V2I and V2V connections.
  • Figure 2: Two types of RL-based system architectures in VFC; fog-level agents (a) and edge-level agents (b).
  • Figure 3: Downtown of Salt Lake City, U.S., showcasing its grid-like structure of 200 by 200 meter city blocks (a) (Map data @2024 Google). Simulated environment based on Salt Lake City blocks (b).
  • Figure 4: VFC modeled as a queuing network; showcasing the possible paths an offloaded task can take to finish processing.
  • Figure 5: Average task delay of various offloading methods; each column is the mean of 100 independent runs.
  • ...and 1 more figures