Sum Rate Maximization in STAR-RIS-UAV-Assisted Networks: A CA-DDPG Approach for Joint Optimization

TL;DR

<3-5 sentence high-level summary> The paper tackles sum-rate maximization in a STAR-RIS-UAV-assisted MU-MISO downlink with joint optimization of BS beamforming, STAR-RIS phase shifts, and UAV placement under non-convex constraints. It introduces CA-DDPG, a convolutionally augmented deep deterministic policy gradient algorithm, augmented with stochastic perturbation to enhance exploration and a CNN-enhanced critic to improve value estimation. Empirical results show CA-DDPG outperforms conventional DDPG, TD3, and DDPG-based DTDE across power regimes and remains robust under imperfect CSI, illustrating strong potential for adaptive 3D placement and RIS control. The work lays a foundation for future extensions to multi-agent settings and integrated sensing and communication applications in next-generation networks.

Abstract

With the rapid advances in programmable materials, reconfigurable intelligent surfaces (RIS) have become a pivotal technology for future wireless communications. The simultaneous transmitting and reflecting reconfigurable intelligent surfaces (STAR-RIS) can both transmit and reflect signals, enabling comprehensive signal control and expanding application scenarios. This paper introduces an unmanned aerial vehicle (UAV) to further enhance system flexibility and proposes an optimization design for the spectrum efficiency of the STAR-RIS-UAV-assisted wireless communication system. We present a deep reinforcement learning (DRL) algorithm capable of iteratively optimizing beamforming, phase shifts, and UAV positioning to maximize the system's sum rate through continuous interactions with the environment. To improve exploration in deterministic policies, we introduce a stochastic perturbation factor, which enhances exploration capabilities. As exploration is strengthened, the algorithm's ability to accurately evaluate the state-action value function becomes critical. Thus, based on the deep deterministic policy gradient (DDPG) algorithm, we propose a convolution-augmented deep deterministic policy gradient (CA-DDPG) algorithm that balances exploration and evaluation to improve the system's sum rate. The simulation results demonstrate that the CA-DDPG algorithm effectively interacts with the environment, optimizing the beamforming matrix, phase shift matrix, and UAV location, thereby improving system capacity and achieving better performance than other algorithms.

Figures (12)

• Figure 1: STAR-RIS-UAV communication system.
• Figure 2: CA-DDPG framework.
• Figure 3: Basic framework of convolution.
• Figure 4: Sum rate of four algorithms under different $P_t$.
• Figure 5: Ablation experiment of the proposed algorithm.