Mobile Cell-Free Massive MIMO with Multi-Agent Reinforcement Learning: A Scalable Framework

TL;DR

This work tackles mobility-enabled cell-free mMIMO by introducing UAV-like mobile-APs and jointly optimizing their movement and downlink power. It develops a scalable MARL framework that combines GNN-aided communication, a permutation-based architecture for dimension reduction, and a directional decoupling strategy to allocate rewards by each agent's contribution. The proposed SF-MADDPG framework, including dynamic and hyper permutation networks plus credit assignment via ARN and HRN, achieves substantial sum SE gains and faster convergence compared to baselines, while reducing observation-space complexity; results indicate linear SE growth with system size and robustness to evolving channels through online learning. These findings highlight a practical, scalable route to high-quality, uniform service in dense, mobile, cell-free networks, with potential impact on 6G-era deployments.

Abstract

Cell-free massive multiple-input multiple-output (mMIMO) offers significant advantages in mobility scenarios, mainly due to the elimination of cell boundaries and strong macro diversity. In this paper, we examine the downlink performance of cell-free mMIMO systems equipped with mobile-APs utilizing the concept of unmanned aerial vehicles, where mobility and power control are jointly considered to effectively enhance coverage and suppress interference. However, the high computational complexity, poor collaboration, limited scalability, and uneven reward distribution of conventional optimization schemes lead to serious performance degradation and instability. These factors complicate the provision of consistent and high-quality service across all user equipments in downlink cell-free mMIMO systems. Consequently, we propose a novel scalable framework enhanced by multi-agent reinforcement learning (MARL) to tackle these challenges. The established framework incorporates a graph neural network (GNN)-aided communication mechanism to facilitate effective collaboration among agents, a permutation architecture to improve scalability, and a directional decoupling architecture to accurately distinguish contributions. In the numerical results, we present comparisons of different optimization schemes and network architectures, which reveal that the proposed scheme can effectively enhance system performance compared to conventional schemes due to the adoption of advanced technologies. In particular, appropriately compressing the observation space of agents is beneficial for achieving a better balance between performance and convergence.

Figures (14)

• Figure 1: Illustration of a cell-free mMIMO system equipped with multi-antenna mobile-APs, which is embedded with a GNN-aided communication architecture to help all mobile-APs aggregate the observed partial information from neighboring mobile-APs and selectively combine received information.
• Figure 2: Illustration of the progressive evolution from a dynamic permutation network to a dynamic hyper permutation network, which consists of four parts: input module $\mathcal{A}$, backbone module $\mathcal{B}$, output module $\mathcal{C}$ for actions $a_\mathrm{inv}$, and output module $\mathcal{D}$ for actions $a_\mathrm{equiv}$.
• Figure 3: The overview of a scalable framework for joint mobility and downlink power control under cell-free mMIMO systems, including a GNN-aided communication architecture, a permutation architecture, and a directional decoupling architecture.
• Figure 4: The two permutation types of actions in the cell-free mMIMO system include permutation equivariant, such as mobility actions and agent power actions, as well as permutation invariant, such as antenna power actions.
• Figure 5: Dynamic trajectories under two different architectures with $M=16$, $K=9$, $N=8$, $\tau_p=K$, and $\Delta = \lambda/2$, including the single PI architecture in Fig. 5 (a), where the black solid arrow represents the movement trajectory of each mobile-AP, and the joint PI and PE architecture in Fig. 5 (b), where the black dashed arrow represents the initial movement trajectory of each mobile-AP, and the blue solid arrow represents the adjusted movement trajectory.

Theorems & Definitions (9)

• Corollary 1
• Theorem 1
• Theorem 2
• Corollary 2
• Theorem 3
• Remark 1
• Remark 2
• Remark 3
• Remark 4