Large Language Model (LLM)-enabled Reinforcement Learning for Wireless Network Optimization

TL;DR

This work tackles optimization of 6G wireless networks under diverse user demands by integrating large language models (LLMs) with reinforcement learning (RL). It proposes a framework that classifies LLM roles within the RL loop—feature extraction, reward design, policy interpretation, and decision-making—and applies them across physical to application layers. A novel LESR-enabled MARL framework for UAV–satellite service migration uses state representations and semantic reasoning to generate intrinsic rewards and guide decisions, achieving notable gains in simulation. The paper also discusses cross-layer challenges, lessons learned, and future directions for secure, low-overhead, end-to-end LLM-assisted RL in wireless networks.

Abstract

Enhancing future wireless networks presents a significant challenge for networking systems due to diverse user demands and the emergence of 6G technology. While reinforcement learning (RL) is a powerful framework, it often encounters difficulties with high-dimensional state spaces and complex environments, leading to substantial computational demands, distributed intelligence, and potentially inconsistent outcomes. Large language models (LLMs), with their extensive pretrained knowledge and advanced reasoning capabilities, offer promising tools to enhance RL in optimizing 6G wireless networks. We explore RL models augmented by LLMs, emphasizing their roles and the potential benefits of their synergy in wireless network optimization. We then examine LLM-enabled RL across various protocol layers: physical, data link, network, transport, and application layers. Additionally, we propose an LLM-assisted state representation and semantic extraction to enhance the multi-agent reinforcement learning (MARL) framework. This approach is applied to service migration and request routing, as well as topology graph generation in unmanned aerial vehicle (UAV)-satellite networks. Through case studies, we demonstrate that our framework effectively performs optimization of wireless network. Finally, we outline prospective research directions for LLM-enabled RL in wireless network optimization.

Figures (5)

• Figure 1: Motivation for integrating LLMs with RL for wireless network optimization. The yellow arrows represent the 6G network provides its current state with LLMs and a subsequent reward signal to the RL agent.
• Figure 2: LLMs-enabled RL for wireless network with four roles.
• Figure 3: Summary of typical LLMs-enabled RL for wireless network with LLMs as feature extractor, reward designer, policy interpreter, decision-maker.
• Figure 4: LLMs-enabled RL for service migration, consisting of four function: (1) UAV-satellite network environment; (2) LLM-enabled RL with prompt template, state representation and intrinsic reward, and feedback; (3) Message Passing: message-passed observation from the recurrent model; (4) RL agent with deep Q network.
• Figure 5: Performance evaluation of our proposed LLMs-enabled MARL with recurrent framework.