Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Doubly-Dynamic ISAC Precoding for Vehicular Networks: A Constrained Deep Reinforcement Learning (CDRL) Approach

Zonghui Yang, Shijian Gao, Xiang Cheng

TL;DR

This work proposes using constrained deep reinforcement learning to facilitate dynamic updates to the ISAC precoder using the primal dual-deep deterministic policy gradient and Wolpertinger architecture, which has superiority over existing candidates and is validated through experiments.

Abstract

Integrated sensing and communication (ISAC) technology is essential for supporting vehicular networks. However, the communication channel in this scenario exhibits time variations, and the potential targets may move rapidly, resulting in double dynamics. This nature poses a challenge for real-time precoder design. While optimization-based solutions are widely researched, they are complex and heavily rely on perfect channel-related information, which is impractical in double dynamics. To address this challenge, we propose using constrained deep reinforcement learning to facilitate dynamic updates to the ISAC precoder. Additionally, the primal dual-deep deterministic policy gradient and Wolpertinger architecture are tailored to efficiently train the algorithm under complex constraints and varying numbers of users. The proposed scheme not only adapts to the dynamics based on observations but also leverages environmental information to enhance performance and reduce complexity. Its superiority over existing candidates has been validated through experiments.

Doubly-Dynamic ISAC Precoding for Vehicular Networks: A Constrained Deep Reinforcement Learning (CDRL) Approach

TL;DR

This work proposes using constrained deep reinforcement learning to facilitate dynamic updates to the ISAC precoder using the primal dual-deep deterministic policy gradient and Wolpertinger architecture, which has superiority over existing candidates and is validated through experiments.

Abstract

Integrated sensing and communication (ISAC) technology is essential for supporting vehicular networks. However, the communication channel in this scenario exhibits time variations, and the potential targets may move rapidly, resulting in double dynamics. This nature poses a challenge for real-time precoder design. While optimization-based solutions are widely researched, they are complex and heavily rely on perfect channel-related information, which is impractical in double dynamics. To address this challenge, we propose using constrained deep reinforcement learning to facilitate dynamic updates to the ISAC precoder. Additionally, the primal dual-deep deterministic policy gradient and Wolpertinger architecture are tailored to efficiently train the algorithm under complex constraints and varying numbers of users. The proposed scheme not only adapts to the dynamics based on observations but also leverages environmental information to enhance performance and reduce complexity. Its superiority over existing candidates has been validated through experiments.
Paper Structure (18 sections, 21 equations, 4 figures, 1 table, 1 algorithm)

This paper contains 18 sections, 21 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model
  3. Transmitted Waveform
  4. Communication Model
  5. Sensing Model
  6. Optimization-based ISAC precoding with perfect prior
  7. Constrained DRL-based Precoding for ISAC without perfect prior
  8. Constrained Markov Decision Process Formulation
  9. State Space
  10. Action space
  11. Reward and cost functions
  12. State transition
  13. Learning Algorithm
  14. Learning architecture
  15. Action selection
  16. ...and 3 more sections

Figures (4)

  • Figure 1: An illustration of the ISAC system in doubly-dynamic scenarios.
  • Figure 2: The proposed CDRL-based framework for ISAC precoding.
  • Figure 3: Cumulative reward and cost versus epochs.
  • Figure 4: ISAC performance comparison among various schemes.