Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lyapunov-guided Deep Reinforcement Learning for Semantic-aware AoI Minimization in UAV-assisted Wireless Networks

Yusi Long, Shimin Gong, Sumei Sun, Gary Lee, Lanhua Li, Dusit Niyato

TL;DR

This paper investigates an unmanned aerial vehicle (UAV) assisted semantic network where the ground users (GUs) periodically capture and upload the sensing information to a base station (BS) via UAVs’ relaying via the Lyapunov framework and uses hierarchical deep reinforcement learning (DRL) to solve each subproblem.

Abstract

This paper investigates an unmanned aerial vehicle (UAV)-assisted semantic network where the ground users (GUs) periodically capture and upload the sensing information to a base station (BS) via UAVs' relaying. Both the GUs and the UAVs can extract semantic information from large-size raw data and transmit it to the BS for recovery. Smaller-size semantic information reduces latency and improves information freshness, while larger-size semantic information enables more accurate data reconstruction at the BS, preserving the value of original information. We introduce a novel semantic-aware age-of-information (SAoI) metric to capture both information freshness and semantic importance, and then formulate a time-averaged SAoI minimization problem by jointly optimizing the UAV-GU association, the semantic extraction, and the UAVs' trajectories. We decouple the original problem into a series of subproblems via the Lyapunov framework and then use hierarchical deep reinforcement learning (DRL) to solve each subproblem. Specifically, the UAV-GU association is determined by DRL, followed by the optimization module updating the semantic extraction strategy and UAVs' deployment. Simulation results show that the hierarchical structure improves learning efficiency. Moreover, it achieves low AoI through semantic extraction while ensuring minimal loss of original information, outperforming the existing baselines.

Lyapunov-guided Deep Reinforcement Learning for Semantic-aware AoI Minimization in UAV-assisted Wireless Networks

TL;DR

This paper investigates an unmanned aerial vehicle (UAV) assisted semantic network where the ground users (GUs) periodically capture and upload the sensing information to a base station (BS) via UAVs’ relaying via the Lyapunov framework and uses hierarchical deep reinforcement learning (DRL) to solve each subproblem.

Abstract

This paper investigates an unmanned aerial vehicle (UAV)-assisted semantic network where the ground users (GUs) periodically capture and upload the sensing information to a base station (BS) via UAVs' relaying. Both the GUs and the UAVs can extract semantic information from large-size raw data and transmit it to the BS for recovery. Smaller-size semantic information reduces latency and improves information freshness, while larger-size semantic information enables more accurate data reconstruction at the BS, preserving the value of original information. We introduce a novel semantic-aware age-of-information (SAoI) metric to capture both information freshness and semantic importance, and then formulate a time-averaged SAoI minimization problem by jointly optimizing the UAV-GU association, the semantic extraction, and the UAVs' trajectories. We decouple the original problem into a series of subproblems via the Lyapunov framework and then use hierarchical deep reinforcement learning (DRL) to solve each subproblem. Specifically, the UAV-GU association is determined by DRL, followed by the optimization module updating the semantic extraction strategy and UAVs' deployment. Simulation results show that the hierarchical structure improves learning efficiency. Moreover, it achieves low AoI through semantic extraction while ensuring minimal loss of original information, outperforming the existing baselines.
Paper Structure (21 sections, 2 theorems, 38 equations, 10 figures, 1 algorithm)

This paper contains 21 sections, 2 theorems, 38 equations, 10 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. UAV-assisted AoI Reduction
  4. UAV-assisted Semantic Communication
  5. Semantic-aware AoI Minimization
  6. System model
  7. Extraction-then-Transmission (ETH) Protocol
  8. Semantic Information Extraction
  9. Information Uploading and Forwarding
  10. Semantic Recovery
  11. Lyapunov decomposition for SAoI Minimization
  12. AoI Dynamics and Semantic Information Value
  13. SAoI Minimization and Lyapunov Decomposition
  14. Per-Slot association, semantic extraction, and UAVs' deployment
  15. PPO for the UAV-GU Association
  16. ...and 6 more sections

Key Result

Proposition 1

For each GU-$k$, $k \in \mathcal{K}$, we can establish a virtual queue $Q_k(i)$ with initial zero state, i.e., $Q_k(0)=0$, and the queue dynamics given by: If $Q_k(i)$ is mean rate stable, i.e., $\lim_{i\rightarrow\infty}\frac{E[|Q_k(i)|]}{i}=0$, we can ensure the satisfaction of the inequality in con:age.

Figures (10)

  • Figure 1: A UAV-assisted semantic communication network
  • Figure 2: The GU-$k$'s AoI dynamics
  • Figure 3: Lya-HiPPO algorithm framework
  • Figure 4: Convergence in a single time slot for model-free PPO and model-based optimization
  • Figure 5: Convergence of the Lya-HiPPO algorithm: single slot with varying number of GUs and multi-slot with fixed number of GUs
  • ...and 5 more figures

Theorems & Definitions (2)

  • Proposition 1
  • Proposition 2