Decentralization of Generative AI via Mixture of Experts for Wireless Networks: A Comprehensive Survey

TL;DR

This survey analyzes how Mixture of Experts (MoE) can decentralize Generative AI (GenAI) in wireless networks, addressing resource constraints and heterogeneity. It synthesizes MoE fundamentals (experts, gating, output aggregation), gating variants (dense, sparse, hard, hierarchical), and integrations with GenAI and RL, then maps these ideas to broad wireless domains including vehicular, UAV, satellite, HetNets, ISAC, and mobile edge computing. A case study demonstrates MoE's gains when integrated into a diffusion-based DRL framework for network optimization, and the paper catalogs open-source datasets across NLP, vision, multimodal, and wireless tasks to support MoE research. The authors conclude with future directions toward lightweight architectures, dynamic and load-balanced gating, cross-layer optimization, hardware acceleration, and incorporation of IoT and 6G paradigms, highlighting MoE as a key enabler for scalable, adaptive, and efficient next-generation wireless systems.

Abstract

Mixture of Experts (MoE) has emerged as a promising paradigm for scaling model capacity while preserving computational efficiency, particularly in large-scale machine learning architectures such as large language models (LLMs). Recent advances in MoE have facilitated its adoption in wireless networks to address the increasing complexity and heterogeneity of modern communication systems. This paper presents a comprehensive survey of the MoE framework in wireless networks, highlighting its potential in optimizing resource efficiency, improving scalability, and enhancing adaptability across diverse network tasks. We first introduce the fundamental concepts of MoE, including various gating mechanisms and the integration with generative AI (GenAI) and reinforcement learning (RL). Subsequently, we discuss the extensive applications of MoE across critical wireless communication scenarios, such as vehicular networks, unmanned aerial vehicles (UAVs), satellite communications, heterogeneous networks, integrated sensing and communication (ISAC), and mobile edge networks. Furthermore, key applications in channel prediction, physical layer signal processing, radio resource management, network optimization, and security are thoroughly examined. Additionally, we present a detailed overview of open-source datasets that are widely used in MoE-based models to support diverse machine learning tasks. Finally, this survey identifies crucial future research directions for MoE, emphasizing the importance of advanced training techniques, resource-aware gating strategies, and deeper integration with emerging 6G technologies.

Figures (12)

• Figure 1: The structure of this survey.
• Figure 2: A comprehensive timeline illustrating the evolution of MoE models, structured across four dimensions: expert networks, gating mechanisms, MoE integrated with GenAI, and MoE applications. The timeline presents diverse expert networks such as FNNs, SVMs, MLPs, CNNs, and self-attention heads, along with the development of dense, sparse, hard, and other variant gating mechanisms. The framework of MoE is widely integrated with GenAI models, such as GANs, VAEs, DMs, and Transformers. Recent applications span NLP, computer vision, multimodal tasks, and wireless networks, demonstrating the versatility of MoE in modern AI and communication systems.
• Figure 3: Illustration of three learning architectures. (a) Classical DNN model: A straightforward architecture for uniform data distribution but limited by low computational and energy efficiency. (b) MoE framework: Incorporate multiple expert networks coordinated by a gating mechanism for diverse data or tasks. (c) MoE with GenAI: Combine the specialization of MoE with generative capability, illustrated by an example of MoE integrated with multiple encoder experts of variational autoencoder (VAE) saidutta2021joint. (d) MoE with RL: Employ multiple actor experts (AEs) and critic experts (CEs) to enhance the training performance of the actor-critic (AC) algorithm zheng2019self.
• Figure 4: Illustration of MoE integration across diverse wireless network scenarios, including vehicular networks, UAV networks, heterogeneous networks (HetNets), satellite networks, integrated sensing and communications (ISAC), and mobile edge networks.
• Figure 5: Illustration of channel estimation using MoE. (a) The MoE framework in lopez2020channel combines a LocB expert that uses the transmitter's and receiver's locations and a LocF expert based on time-of-arrival features, weighted by a gating network optimized through block-coordinate minimization. (b) Exploit transfer learning to fine-tune a pre-trained MoE model or directly train a new MoE network without model transfer jaiswal2023leveraging. For the transfer learning scheme, a CNN module is employed to compute the similarity between the source and new environments, determining the amount of data required for fine-tuning the pre-trained MoE.