Wireless MAC Protocol Synthesis and Optimization with Multi-Agent Distributed Reinforcement Learning

TL;DR

This framework is the first of a kind that enables distributed multi-agent learning within the ns-3 environment, and facilitates the design and synthesis of adaptive MAC protocols tailored to specific environmental conditions.

Abstract

In this letter, we propose a novel Multi-Agent Deep Reinforcement Learning (MADRL) framework for Medium Access Control (MAC) protocol design. Unlike centralized approaches, which rely on a single entity for decision-making, MADRL empowers individual network nodes to autonomously learn and optimize their MAC based on local observations. Leveraging ns3-ai and RLlib, as far as we are aware of, our framework is the first of a kind that enables distributed multi-agent learning within the ns-3 environment, facilitating the design and synthesis of adaptive MAC protocols tailored to specific environmental conditions. We demonstrate the effectiveness of the MADRL MAC framework through extensive simulations, showcasing superior performance compared to legacy protocols across diverse scenarios. Our findings highlight the potential of MADRL-based MAC protocols to significantly enhance Quality of Service (QoS) requirements for future wireless applications.

Figures (6)

• Figure 1: System concept.
• Figure 2: NR-U LBT4 Channel access mechanism.
• Figure 3: Distributed training and execution architecture.
• Figure 4: Integration of ns3-ai gym with RLlib using a dummy environment.
• Figure 5: Learning curves comparing the convergence of proposed DTDE against the centralized approach CTCE.