Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM4MAC: An LLM-Driven Reinforcement Learning Framework for MAC Protocol Emergence

Renxuan Tan, Rongpeng Li, Zhifeng Zhao

TL;DR

Extensive simulations demonstrate that on top of a compact LLM, which is purposefully selected to balance performance with resource efficiency, the protocol emerging from LLM4MAC outperforms comparative baselines in throughput and generalization.

Abstract

With the advent of 6G systems, emerging hyper-connected ecosystems necessitate agile and adaptive medium access control (MAC) protocols to contend with network dynamics and diverse service requirements. We propose LLM4MAC, a novel framework that harnesses large language models (LLMs) within a reinforcement learning paradigm to drive MAC protocol emergence. By reformulating uplink data transmission scheduling as a semantics-generalized partially observable Markov game (POMG), LLM4MAC encodes network operations in natural language, while proximal policy optimization (PPO) ensures continuous alignment with the evolving network dynamics. A structured identity embedding (SIE) mechanism further enables robust coordination among heterogeneous agents. Extensive simulations demonstrate that on top of a compact LLM, which is purposefully selected to balance performance with resource efficiency, the protocol emerging from LLM4MAC outperforms comparative baselines in throughput and generalization.

LLM4MAC: An LLM-Driven Reinforcement Learning Framework for MAC Protocol Emergence

TL;DR

Extensive simulations demonstrate that on top of a compact LLM, which is purposefully selected to balance performance with resource efficiency, the protocol emerging from LLM4MAC outperforms comparative baselines in throughput and generalization.

Abstract

With the advent of 6G systems, emerging hyper-connected ecosystems necessitate agile and adaptive medium access control (MAC) protocols to contend with network dynamics and diverse service requirements. We propose LLM4MAC, a novel framework that harnesses large language models (LLMs) within a reinforcement learning paradigm to drive MAC protocol emergence. By reformulating uplink data transmission scheduling as a semantics-generalized partially observable Markov game (POMG), LLM4MAC encodes network operations in natural language, while proximal policy optimization (PPO) ensures continuous alignment with the evolving network dynamics. A structured identity embedding (SIE) mechanism further enables robust coordination among heterogeneous agents. Extensive simulations demonstrate that on top of a compact LLM, which is purposefully selected to balance performance with resource efficiency, the protocol emerging from LLM4MAC outperforms comparative baselines in throughput and generalization.

Paper Structure

This paper contains 16 sections, 12 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. System Model
  4. Background on UDTS
  5. Delivery Model
  6. Channel Model
  7. Decision Model
  8. Semantic-Generalized POMG Formulation
  9. PPO-based LLM Decision Maker for Protocol Emergence
  10. Preliminaries on PPO
  11. LLM-Driven Protocol Emergence with PPO Alignment
  12. SIE-Enhanced Prompt
  13. LLM-based Multi-Agent Policy
  14. PPO Alignment
  15. Simulation Results and Analysis
  16. ...and 1 more sections

Figures (5)

  • Figure 1: LLM4MAC framework consists of three main components: prompt formulation transforms observations into identifiable model inputs, decision maker determines actions, and environmental feedback with PPO for functional alignment.
  • Figure 2: Numeric-Text transformation and prompt assembly.
  • Figure 3: The comparisons of network throughput with other algorithms, where five independent experiments are performed for each case.
  • Figure 4: Rewards v.s. training steps, where rewards are evaluated after each indicated training step. Experiments are conducted with different random seeds.
  • Figure 5: The comparisons in terms of response latency (evaluated on Nvidia RTX 3090 GPU), valid answer percentage, and average environment reward.