Enhancement of High-definition Map Update Service Through Coverage-aware and Reinforcement Learning
Jeffrey Redondo, Zhenhui Yuan, Nauman Aslam
TL;DR
HD map updates in autonomous vehicle networks face stringent latency and high data‑rate demands under dynamic VANET topologies. The paper proposes a nonintrusive, application‑layer Q‑learning approach that uses sojourn time to inform transmission waiting times, with a utility‑based reward comprising normalized throughput, latency penalties, and bonuses to mimic EDCA behavior. Key contributions include a compact state representation $\\{S_j, T_v, C, T_{cv}\}$, a category‑dependent action mapping, sojourn time discretization within a 200 m RSU range, and a TD‑Q learning rule $Q(s,a)=Q(s,a)+\alpha\big[r+\gamma\max_{a'}Q(s',a')-Q(s,a)\big]$, all validated in a VANET simulation against IEEE802.11p and IEEE802.11ac baselines. Results show substantial latency reductions for HD map updates (up to $75\%$ vs no QoS and $73\%$ vs QoS), improved throughput, and maintained fairness, indicating that application‑layer RL can deliver real‑time HD map QoS without altering MAC protocols. The work also demonstrates compatibility with IEEE802.11ac and points to future multi‑agent extensions and the exploration of newer wireless technologies and dynamic packet sizing.
Abstract
High-definition (HD) Map systems will play a pivotal role in advancing autonomous driving to a higher level, thanks to the significant improvement over traditional two-dimensional (2D) maps. Creating an HD Map requires a huge amount of on-road and off-road data. Typically, these raw datasets are collected and uploaded to cloud-based HD map service providers through vehicular networks. Nevertheless, there are challenges in transmitting the raw data over vehicular wireless channels due to the dynamic topology. As the number of vehicles increases, there is a detrimental impact on service quality, which acts as a barrier to a real-time HD Map system for collaborative driving in Autonomous Vehicles (AV). In this paper, to overcome network congestion, a Q-learning coverage-time-awareness algorithm is presented to optimize the quality of service for vehicular networks and HD map updates. The algorithm is evaluated in an environment that imitates a dynamic scenario where vehicles enter and leave. Results showed an improvement in latency for HD map data of $75\%$, $73\%$, and $10\%$ compared with IEEE802.11p without Quality of Service (QoS), IEEE802.11 with QoS, and IEEE802.11p with new access category (AC) for HD map, respectively.