Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimization of End-to-End AoI in Edge-Enabled Vehicular Fog Systems: A Dueling-DQN Approach

Seifu Birhanu Tadele, Binayak Kar, Frezer Guteta Wakgra, Asif Uddin Khan

TL;DR

This work tackles end-to-end AoI in an edge-enabled vehicular fog system by explicitly accounting for both source-to-edge and edge-to-destination transmissions, a gap in prior literature. It develops a queuing-theoretic model of IoT sources, edge servers, and vehicular fogs, derives expressions for transmission and waiting times, and formulates the objective to minimize the average end-to-end AoI $\Delta$. A modified dueling-DQN DRL policy is proposed to learn optimal offloading decisions, with states capturing arrivals, computing capacities, and AoI, actions encoding binary offloads, and rewards tied to AoI performance. Experiments show that the dueling-DQN approach converges faster and achieves lower end-to-end AoI than a standard DQN and analytical baselines, demonstrating improved decision freshness for real-time ITS applications.

Abstract

In real-time status update services for the Internet of Things (IoT), the timely dissemination of information requiring timely updates is crucial to maintaining its relevance. Failing to keep up with these updates results in outdated information. The age of information (AoI) serves as a metric to quantify the freshness of information. The Existing works to optimize AoI primarily focus on the transmission time from the information source to the monitor, neglecting the transmission time from the monitor to the destination. This oversight significantly impacts information freshness and subsequently affects decision-making accuracy. To address this gap, we designed an edge-enabled vehicular fog system to lighten the computational burden on IoT devices. We examined how information transmission and request-response times influence end-to-end AoI. As a solution, we proposed Dueling-Deep Queue Network (dueling-DQN), a deep reinforcement learning (DRL)-based algorithm and compared its performance with DQN policy and analytical results. Our simulation results demonstrate that the proposed dueling-DQN algorithm outperforms both DQN and analytical methods, highlighting its effectiveness in improving real-time system information freshness. Considering the complete end-to-end transmission process, our optimization approach can improve decision-making performance and overall system efficiency.

Optimization of End-to-End AoI in Edge-Enabled Vehicular Fog Systems: A Dueling-DQN Approach

TL;DR

This work tackles end-to-end AoI in an edge-enabled vehicular fog system by explicitly accounting for both source-to-edge and edge-to-destination transmissions, a gap in prior literature. It develops a queuing-theoretic model of IoT sources, edge servers, and vehicular fogs, derives expressions for transmission and waiting times, and formulates the objective to minimize the average end-to-end AoI . A modified dueling-DQN DRL policy is proposed to learn optimal offloading decisions, with states capturing arrivals, computing capacities, and AoI, actions encoding binary offloads, and rewards tied to AoI performance. Experiments show that the dueling-DQN approach converges faster and achieves lower end-to-end AoI than a standard DQN and analytical baselines, demonstrating improved decision freshness for real-time ITS applications.

Abstract

In real-time status update services for the Internet of Things (IoT), the timely dissemination of information requiring timely updates is crucial to maintaining its relevance. Failing to keep up with these updates results in outdated information. The age of information (AoI) serves as a metric to quantify the freshness of information. The Existing works to optimize AoI primarily focus on the transmission time from the information source to the monitor, neglecting the transmission time from the monitor to the destination. This oversight significantly impacts information freshness and subsequently affects decision-making accuracy. To address this gap, we designed an edge-enabled vehicular fog system to lighten the computational burden on IoT devices. We examined how information transmission and request-response times influence end-to-end AoI. As a solution, we proposed Dueling-Deep Queue Network (dueling-DQN), a deep reinforcement learning (DRL)-based algorithm and compared its performance with DQN policy and analytical results. Our simulation results demonstrate that the proposed dueling-DQN algorithm outperforms both DQN and analytical methods, highlighting its effectiveness in improving real-time system information freshness. Considering the complete end-to-end transmission process, our optimization approach can improve decision-making performance and overall system efficiency.
Paper Structure (40 sections, 4 theorems, 53 equations, 9 figures, 4 tables, 1 algorithm)

This paper contains 40 sections, 4 theorems, 53 equations, 9 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. AoI Optimization using Queue Models
  4. AoI Optimization based on Edge-enabled Models
  5. AoI Optimization using RL Approach
  6. System Model and Analysis of AoI
  7. Edge-enabled Vehicular Fog Model
  8. AoI Analysis in Edge-enabled Vehicular Fog Systems
  9. Communication model
  10. Time-Average Age Analysis
  11. Analysis of Packet transmission rate
  12. Analysis of $\mathbb{E}\left[I_j^T\right]$
  13. Analysis of $\mathbb{E}\left[V_j^T\right]$
  14. Analysis of $\mathbb{E}\left[L_j^P\right]$
  15. Analysis of $\mathbb{E}\left[L_j^T\right]$
  16. ...and 25 more sections

Key Result

Theorem 1

The time it takes to successfully transmit packets generated by $j$th IoT device is calculated as

Figures (9)

  • Figure 1: Illustration of the edge and vehicular fog architecture with the queueing system for a given packet management policy.
  • Figure 2: An example of how AoI evolves for $j$th IoT devices. Triangles and circles represent packet arrival times at edge nodes and vehicular fog, respectively.
  • Figure 3: The exchange of information between the environment and the proposed dueling-DQN algorithm.
  • Figure 4: The evaluations of convergence of DRL agents.
  • Figure 5: Evaluation of the dueling-DQN with the increasing number of IoT devices and time slots over the episodes
  • ...and 4 more figures

Theorems & Definitions (9)

  • Definition 1
  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Theorem 4
  • proof