Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-User Beamforming with Deep Reinforcement Learning in Sensing-Aided Communication

Xiyu Wang, Gilberto Berardinelli, Hei Victor Cheng, Petar Popovski, Ramoni Adeogun

TL;DR

This work tackles beam drift in multi-user mmWave ISAC by leveraging sensing echoes to dynamically manage multiple beams without user feedback. It introduces a DRL-based policy (PPO) that optimizes discrete user-type decisions and beam allocations to maximize frame throughput, contrasted with an AoD-based heuristic and conventional beam sweeping. The approach uses a sensing-centric state representation and a reduced action space to achieve robust performance across varying user speeds, outperforming baselines and approaching an AoD-Genie upper bound. The results demonstrate that sensing-driven beam management can significantly enhance throughput in dynamic vehicular scenarios, with implications for scalable MAC-layer design in ISAC systems.

Abstract

Mobile users are prone to experience beam failure due to beam drifting in millimeter wave (mmWave) communications. Sensing can help alleviate beam drifting with timely beam changes and low overhead since it does not need user feedback. This work studies the problem of optimizing sensing-aided communication by dynamically managing beams allocated to mobile users. A multi-beam scheme is introduced, which allocates multiple beams to the users that need an update on the angle of departure (AoD) estimates and a single beam to the users that have satisfied AoD estimation precision. A deep reinforcement learning (DRL) assisted method is developed to optimize the beam allocation policy, relying only upon the sensing echoes. For comparison, a heuristic AoD-based method using approximated Cramér-Rao lower bound (CRLB) for allocation is also presented. Both methods require neither user feedback nor prior state evolution information. Results show that the DRL-assisted method achieves a considerable gain in throughput than the conventional beam sweeping method and the AoD-based method, and it is robust to different user speeds.

Multi-User Beamforming with Deep Reinforcement Learning in Sensing-Aided Communication

TL;DR

This work tackles beam drift in multi-user mmWave ISAC by leveraging sensing echoes to dynamically manage multiple beams without user feedback. It introduces a DRL-based policy (PPO) that optimizes discrete user-type decisions and beam allocations to maximize frame throughput, contrasted with an AoD-based heuristic and conventional beam sweeping. The approach uses a sensing-centric state representation and a reduced action space to achieve robust performance across varying user speeds, outperforming baselines and approaching an AoD-Genie upper bound. The results demonstrate that sensing-driven beam management can significantly enhance throughput in dynamic vehicular scenarios, with implications for scalable MAC-layer design in ISAC systems.

Abstract

Mobile users are prone to experience beam failure due to beam drifting in millimeter wave (mmWave) communications. Sensing can help alleviate beam drifting with timely beam changes and low overhead since it does not need user feedback. This work studies the problem of optimizing sensing-aided communication by dynamically managing beams allocated to mobile users. A multi-beam scheme is introduced, which allocates multiple beams to the users that need an update on the angle of departure (AoD) estimates and a single beam to the users that have satisfied AoD estimation precision. A deep reinforcement learning (DRL) assisted method is developed to optimize the beam allocation policy, relying only upon the sensing echoes. For comparison, a heuristic AoD-based method using approximated Cramér-Rao lower bound (CRLB) for allocation is also presented. Both methods require neither user feedback nor prior state evolution information. Results show that the DRL-assisted method achieves a considerable gain in throughput than the conventional beam sweeping method and the AoD-based method, and it is robust to different user speeds.
Paper Structure (14 sections, 18 equations, 4 figures, 1 table)

This paper contains 14 sections, 18 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. System Model
  3. Transmit Signal
  4. Communication Model
  5. Sensing Model
  6. Sensing - AoD Estimation
  7. Beam Management Problem
  8. Objective Function
  9. AoD-Based Method
  10. User and Beam allocation
  11. Power allocation
  12. DRL-assisted Beam Management
  13. Simulation Results
  14. Conclusion

Figures (4)

  • Figure 1: (a) An illustration of considered multi-user ISAC scenario. (b) A snapshot of user trajectory in one frame.
  • Figure 2: Sensing-Assisted Communication Protocol
  • Figure 3: CDF of normalized throughput when users averaged speed is 20 m/s
  • Figure 4: Comparison of averaged throughput as a function of user speed among different methods