Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Coverage-aware and Reinforcement Learning Using Multi-agent Approach for HD Map QoS in a Realistic Environment

Jeffrey Redondo, Zhenhui Yuan, Nauman Aslam, Juan Zhang

TL;DR

This work has proposed a Q- Learning algorithm that operates at the application layer that has demonstrated a better network performance with relatively fewer optimization requirements as compared to the Deep Q Network (DQN) and Actor-Critic algorithms.

Abstract

One effective way to optimize the offloading process is by minimizing the transmission time. This is particularly true in a Vehicular Adhoc Network (VANET) where vehicles frequently download and upload High-definition (HD) map data which requires constant updates. This implies that latency and throughput requirements must be guaranteed by the wireless system. To achieve this, adjustable contention windows (CW) allocation strategies in the standard IEEE802.11p have been explored by numerous researchers. Nevertheless, their implementations demand alterations to the existing standard which is not always desirable. To address this issue, we proposed a Q-Learning algorithm that operates at the application layer. Moreover, it could be deployed in any wireless network thereby mitigating the compatibility issues. The solution has demonstrated a better network performance with relatively fewer optimization requirements as compared to the Deep Q Network (DQN) and Actor-Critic algorithms. The same is observed while evaluating the model in a multi-agent setup showing higher performance compared to the single-agent setup.

Coverage-aware and Reinforcement Learning Using Multi-agent Approach for HD Map QoS in a Realistic Environment

TL;DR

This work has proposed a Q- Learning algorithm that operates at the application layer that has demonstrated a better network performance with relatively fewer optimization requirements as compared to the Deep Q Network (DQN) and Actor-Critic algorithms.

Abstract

One effective way to optimize the offloading process is by minimizing the transmission time. This is particularly true in a Vehicular Adhoc Network (VANET) where vehicles frequently download and upload High-definition (HD) map data which requires constant updates. This implies that latency and throughput requirements must be guaranteed by the wireless system. To achieve this, adjustable contention windows (CW) allocation strategies in the standard IEEE802.11p have been explored by numerous researchers. Nevertheless, their implementations demand alterations to the existing standard which is not always desirable. To address this issue, we proposed a Q-Learning algorithm that operates at the application layer. Moreover, it could be deployed in any wireless network thereby mitigating the compatibility issues. The solution has demonstrated a better network performance with relatively fewer optimization requirements as compared to the Deep Q Network (DQN) and Actor-Critic algorithms. The same is observed while evaluating the model in a multi-agent setup showing higher performance compared to the single-agent setup.
Paper Structure (21 sections, 17 equations, 7 figures, 2 tables)

This paper contains 21 sections, 17 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Contention Windows & EDCA
  5. Problem Statement
  6. Design
  7. Reinforcement Learning
  8. DQN
  9. Actor-Critic
  10. State, Action, Reward
  11. State
  12. Categories
  13. Actions
  14. Reward
  15. Simulation
  16. ...and 6 more sections

Figures (7)

  • Figure 1: Latency comparison between solution with and without penalties using different packet size. (a) HD Map, (b) Video, (c) Best-Effort, and (d) Voice.
  • Figure 2: Throughput comparison between solution with and without penalties using different packet size. (a) HD Map, (b) Video, (c) Best-Effort, and (d) Voice.
  • Figure 3: Comparison with different reinforcement learning algorithms using IEEE802.11p
  • Figure 4: Scenario Multi-RSU.
  • Figure 5: Latency for MultiRSU environment. (a) Voice, (b) Video, (c) HD Map, and (d) Best-Effort.
  • ...and 2 more figures