Parametrized Sharing for Multi-Agent Hybrid DRL for Multiple Multi-Functional RISs-Aided Downlink NOMA Networks
Chi-Te Kuo, Li-Hsiang Shen, Jyun-Jhe Huang
TL;DR
The paper tackles energy-efficiency optimization in downlink NOMA networks aided by multiple multifunctional RISs (MF-RISs) that incorporate reflection, transmission, amplification, and energy harvesting. It introduces PMHRL, a multi-agent hybrid DRL framework that couples PPO for continuous decisions with DQN for discrete actions, enhanced by a parametrized sharing mechanism to improve cross-module coordination. The optimization jointly tunes user powers, BS beamforming, MF-RIS amplification, phase shifts, EH ratios, and MF-RIS positions, under self-sustainability and QoS constraints, with the EE objective defined as $EE = \frac{\sum_{k,j} R_{kj}}{P_{total}}$. Simulation results show PMHRL achieves the highest EE compared to benchmarks, and deploying more MF-RISs with EH capabilities yields notable gains in coverage and efficiency under various MA schemes.
Abstract
Multi-functional reconfigurable intelligent surface (MF-RIS) is conceived to address the communication efficiency thanks to its extended signal coverage from its active RIS capability and self-sustainability from energy harvesting (EH). We investigate the architecture of multi-MF-RISs to assist non-orthogonal multiple access (NOMA) downlink networks. We formulate an energy efficiency (EE) maximization problem by optimizing power allocation, transmit beamforming and MF-RIS configurations of amplitudes, phase-shifts and EH ratios, as well as the position of MF-RISs, while satisfying constraints of available power, user rate requirements, and self-sustainability property. We design a parametrized sharing scheme for multi-agent hybrid deep reinforcement learning (PMHRL), where the multi-agent proximal policy optimization (PPO) and deep-Q network (DQN) handle continuous and discrete variables, respectively. The simulation results have demonstrated that proposed PMHRL has the highest EE compared to other benchmarks, including cases without parametrized sharing, pure PPO and DQN. Moreover, the proposed multi-MF-RISs-aided downlink NOMA achieves the highest EE compared to scenarios of no-EH/amplification, traditional RISs, and deployment without RISs/MF-RISs under different multiple access.