Multi-agent Reinforcement Learning-based Joint Precoding and Phase Shift Optimization for RIS-aided Cell-Free Massive MIMO Systems
Yiyang Zhu, Enyu Shi, Ziheng Liu, Jiayi Zhang, Bo Ai
TL;DR
This work tackles the challenge of maximizing sum spectral efficiency in RIS-aided cell-free mMIMO by jointly optimizing AP precoding and RIS phase shifts. It introduces a centralized-training, distributed-execution MARL framework enhanced with fuzzy logic (FL-MARL), where each AP acts as an agent and phase shifts are learned via a two-layer network using local CSI, reducing backhaul burden. The approach reformulates the non-convex problem into a MARL-CTDE setting and demonstrates that FL-MARL achieves SE performance close to MADDPG while significantly reducing computational complexity and speeding up convergence (about 30% faster in reported tests) compared to conventional methods. Empirical results show that the proposed method scales well with more antennas, RISs, and RIS elements, outperforming alternating optimization and offering practical benefits for large, backhaul-constrained deployments. The findings underscore FL-MARL as a viable, scalable solution for real-world RIS-aided CF mMIMO systems, with future work on STAR-RIS and imperfect CSI scenarios.
Abstract
Cell-free (CF) massive multiple-input multiple-output (mMIMO) is a promising technique for achieving high spectral efficiency (SE) using multiple distributed access points (APs). However, harsh propagation environments often lead to significant communication performance degradation due to high penetration loss. To overcome this issue, we introduce the reconfigurable intelligent surface (RIS) into the CF mMIMO system as a low-cost and power-efficient solution. In this paper, we focus on optimizing the joint precoding design of the RIS-aided CF mMIMO system to maximize the sum SE. This involves optimizing the precoding matrix at the APs and the reflection coefficients at the RIS. To tackle this problem, we propose a fully distributed multi-agent reinforcement learning (MARL) algorithm that incorporates fuzzy logic (FL). Unlike conventional approaches that rely on alternating optimization techniques, our FL-based MARL algorithm only requires local channel state information, which reduces the need for high backhaul capacity. Simulation results demonstrate that our proposed FL-MARL algorithm effectively reduces computational complexity while achieving similar performance as conventional MARL methods.