Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Event-Triggered Reinforcement Learning Based Joint Resource Allocation for Ultra-Reliable Low-Latency V2X Communications

Nasir Khan, Sinem Coleri

TL;DR

The paper addresses the critical URLLC-V2X challenge of reliably delivering short safety messages within tight latency by jointly optimizing transmit power and finite-blocklength block lengths. It jointly develops an optimization-theory solution based on convexity analysis and KKT conditions and a centralized event-triggered DRL framework that learns as needed to reduce computational burden, combining multiple DQNs for block-length decisions with a DDPG policy for power. Empirical results show the event-triggered DRL achieves about 95% of the joint-optimization reliability with up to 24% fewer DRL executions and sub-millisecond online decision latency, highlighting practical viability for dynamic vehicular networks. The work also discusses implementation considerations, constraint satisfaction, and extensions (e.g., primal-dual DRL, GNNs) to enhance scalability and robustness in real deployments.

Abstract

Future 6G-enabled vehicular networks face the challenge of ensuring ultra-reliable low-latency communication (URLLC) for delivering safety-critical information in a timely manner. Existing resource allocation schemes for vehicle-to-everything (V2X) communication systems primarily rely on traditional optimization-based algorithms. However, these methods often fail to guarantee the strict reliability and latency requirements of URLLC applications in dynamic vehicular environments due to the high complexity and communication overhead of the solution methodologies. This paper proposes a novel deep reinforcement learning (DRL) based framework for the joint power and block length allocation to minimize the worst-case decoding-error probability in the finite block length (FBL) regime for a URLLC-based downlink V2X communication system. The problem is formulated as a non-convex mixed-integer nonlinear programming problem (MINLP). Initially, an algorithm grounded in optimization theory is developed based on deriving the joint convexity of the decoding error probability in the block length and transmit power variables within the region of interest. Subsequently, an efficient event-triggered DRL-based algorithm is proposed to solve the joint optimization problem. Incorporating event-triggered learning into the DRL framework enables assessing whether to initiate the DRL process, thereby reducing the number of DRL process executions while maintaining reasonable reliability performance. Simulation results demonstrate that the proposed event-triggered DRL scheme can achieve 95% of the performance of the joint optimization scheme while reducing the DRL executions by up to 24% for different network settings.

Event-Triggered Reinforcement Learning Based Joint Resource Allocation for Ultra-Reliable Low-Latency V2X Communications

TL;DR

The paper addresses the critical URLLC-V2X challenge of reliably delivering short safety messages within tight latency by jointly optimizing transmit power and finite-blocklength block lengths. It jointly develops an optimization-theory solution based on convexity analysis and KKT conditions and a centralized event-triggered DRL framework that learns as needed to reduce computational burden, combining multiple DQNs for block-length decisions with a DDPG policy for power. Empirical results show the event-triggered DRL achieves about 95% of the joint-optimization reliability with up to 24% fewer DRL executions and sub-millisecond online decision latency, highlighting practical viability for dynamic vehicular networks. The work also discusses implementation considerations, constraint satisfaction, and extensions (e.g., primal-dual DRL, GNNs) to enhance scalability and robustness in real deployments.

Abstract

Future 6G-enabled vehicular networks face the challenge of ensuring ultra-reliable low-latency communication (URLLC) for delivering safety-critical information in a timely manner. Existing resource allocation schemes for vehicle-to-everything (V2X) communication systems primarily rely on traditional optimization-based algorithms. However, these methods often fail to guarantee the strict reliability and latency requirements of URLLC applications in dynamic vehicular environments due to the high complexity and communication overhead of the solution methodologies. This paper proposes a novel deep reinforcement learning (DRL) based framework for the joint power and block length allocation to minimize the worst-case decoding-error probability in the finite block length (FBL) regime for a URLLC-based downlink V2X communication system. The problem is formulated as a non-convex mixed-integer nonlinear programming problem (MINLP). Initially, an algorithm grounded in optimization theory is developed based on deriving the joint convexity of the decoding error probability in the block length and transmit power variables within the region of interest. Subsequently, an efficient event-triggered DRL-based algorithm is proposed to solve the joint optimization problem. Incorporating event-triggered learning into the DRL framework enables assessing whether to initiate the DRL process, thereby reducing the number of DRL process executions while maintaining reasonable reliability performance. Simulation results demonstrate that the proposed event-triggered DRL scheme can achieve 95% of the performance of the joint optimization scheme while reducing the DRL executions by up to 24% for different network settings.
Paper Structure (17 sections, 1 theorem, 18 equations, 8 figures, 4 tables, 2 algorithms)

This paper contains 17 sections, 1 theorem, 18 equations, 8 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions and organization
  3. System Model and Assumptions
  4. Worst-case Decoding Error Probability Minimization Problem
  5. Joint Optimization of Power and block length
  6. Event-Triggered DRL-Based Power and block length allocation Scheme
  7. Deep Reinforcement Learning Overview
  8. DRL-Based Resource Allocation Framework
  9. Event-triggered learning
  10. Proposed DRL-based Resource Allocation Algorithm
  11. Performance Evaluation
  12. Simulation Setup
  13. Analysis of Event-triggered learning
  14. Performance Comparison of Algorithms
  15. Impact of Vehicular Mobility and Reliability Performance:
  16. ...and 2 more sections

Key Result

Lemma 1

Under the assumption that $\gamma_{k}\left(b_{k}\right)\geq \gamma_{th}=0$ dB and the Shannon rate exceeds the coding rate, i.e., $C_{k}\left(\gamma_{k}\left(b_{k}\right)\right)- a_{k}L >0$, the $\cal{OPBP}$ is a convex optimization problem when $m_{k}$ satisfies:

Figures (8)

  • Figure 1: System model for the V2X communication network.
  • Figure 2: Proposed event-triggered DRL framework for power control and block length allocation.
  • Figure 3: Average worst-case decoding-error probability performance of the algorithms, where (a) shows the training performance, (b) shows the testing performance for different event-trigger threshold values $\upsilon =\left\{0.1, 0.3, 0.5, 0.9 \right\}$, and (c) shows the percentage of DRL executions for varying trigger threshold values, with $K=20$, $P_{max}=23$ dBm, and $M_{D}=300$ symbols.
  • Figure 4: Average worst-case decoding-error probability performance of the algorithms.
  • Figure 5: Optimized average worst-case decoding-error probability of the algorithms versus the number of symbols $M_{D}$.
  • ...and 3 more figures

Theorems & Definitions (2)

  • Lemma 1
  • proof