Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantic-aware Transmission Scheduling: a Monotonicity-driven Deep Reinforcement Learning Approach

Jiazheng Chen, Wanchun Liu, Daniel Quevedo, Yonghui Li, Branka Vucetic

TL;DR

This letter establishes the monotonicity of the Q function of the optimal semantic-aware scheduling policy and develops advanced deep reinforcement learning (DRL) algorithms by leveraging the theoretical guideline.

Abstract

For cyber-physical systems in the 6G era, semantic communications connecting distributed devices for dynamic control and remote state estimation are required to guarantee application-level performance, not merely focus on communication-centric performance. Semantics here is a measure of the usefulness of information transmissions. Semantic-aware transmission scheduling of a large system often involves a large decision-making space, and the optimal policy cannot be obtained by existing algorithms effectively. In this paper, we first investigate the fundamental properties of the optimal semantic-aware scheduling policy and then develop advanced deep reinforcement learning (DRL) algorithms by leveraging the theoretical guidelines. Our numerical results show that the proposed algorithms can substantially reduce training time and enhance training performance compared to benchmark algorithms.

Semantic-aware Transmission Scheduling: a Monotonicity-driven Deep Reinforcement Learning Approach

TL;DR

This letter establishes the monotonicity of the Q function of the optimal semantic-aware scheduling policy and develops advanced deep reinforcement learning (DRL) algorithms by leveraging the theoretical guideline.

Abstract

For cyber-physical systems in the 6G era, semantic communications connecting distributed devices for dynamic control and remote state estimation are required to guarantee application-level performance, not merely focus on communication-centric performance. Semantics here is a measure of the usefulness of information transmissions. Semantic-aware transmission scheduling of a large system often involves a large decision-making space, and the optimal policy cannot be obtained by existing algorithms effectively. In this paper, we first investigate the fundamental properties of the optimal semantic-aware scheduling policy and then develop advanced deep reinforcement learning (DRL) algorithms by leveraging the theoretical guidelines. Our numerical results show that the proposed algorithms can substantially reduce training time and enhance training performance compared to benchmark algorithms.
Paper Structure (16 sections, 3 theorems, 31 equations, 3 figures, 2 tables)

This paper contains 16 sections, 3 theorems, 31 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. System Model
  3. Wireless Communication Model
  4. Semantic Communication Performance
  5. Semantic-aware Transmission Scheduling
  6. Markov Decision Process Formulation
  7. Value Functions of the Optimal Policy
  8. DDPG for scheduling policy optimization
  9. Partial Monotonicity of Q Function
  10. Partially Monotonic Critic Neural Network
  11. Network Architecture-enabled Monotonicity
  12. Regularization-based Monotonicity
  13. Positive derivative penalty (type I)
  14. Positive increment penalty (type II)
  15. Numerical Experiments
  16. ...and 1 more sections

Key Result

Lemma 1

Consider states $\mathbf{s} = (\bm{\tau}, \mathbf{H})$ and $\mathbf{s}'_{\text{AoI}} = (\bm{\tau}'_{(n)}, \mathbf{H})$, where $\bm{\tau}'_{(n)} = (\tau_{1}, \dots, \tau'_{n}, \dots, \tau_{N})$ and $\tau'_{n} \geq \tau_{n}$. For any $n\in\{1,\dots,N\}$, the following holds

Figures (3)

  • Figure 1: Partially monotonic critic NNs. (a) A shallow critic NN with network architecture-enabled monotonicity, where the NN has only one hidden layer. (b) A deep critic NN with regularization-based monotonicity.
  • Figure 2: Average sum MSE during training with $N = 6, M = 3$. The critic NN has only one hidden layer -- a shallow NN setup.
  • Figure 3: Average sum MSE during training with $N = 14, M=7$. The critic NN has three hidden layers -- a deep NN setup.

Theorems & Definitions (7)

  • Example 1: A remote state estimation system
  • Remark 1
  • Lemma 1: Monotonicity of V function w.r.t. AoI states chen2022seDRL
  • Theorem 1: Monotonicity of Q function w.r.t. AoI states
  • proof
  • Theorem 2: Monotonicity of Q function w.r.t. channel states
  • proof