Towards Native Intelligence: 6G-LLM Trained with Reinforcement Learning from NDT Feedback

TL;DR

The paper addresses the challenge of building network-native intelligence for 6G by moving beyond offline, human-curated corpora toward continual learning using Reinforcement Learning from Digital Twin Feedback (RLDTF). It frames 6G-LLMs as orchestrators that translate service intents into executable network configurations, guided by digital-twin QoS rewards and a token-importance weighting scheme. Key contributions include a two-stage training pipeline with reject sampling, a tailored reward design balancing QoS satisfaction and resource usage, a perturbation-based token weighting mechanism, and a PPO-style RL algorithm with three-model setup (action, critic, and reference) to ensure stable learning. Experimental results show the approach achieves around 75% one-shot task completion, improved solution quality, and a practical edge-deployable model, highlighting RLDTF as a promising path toward practical native intelligence in future wireless networks.

Abstract

Owing to its comprehensive understanding of upper-layer application requirements and the capabilities of practical communication systems, the 6G-LLM (6G domain large language model) offers a promising pathway toward realizing network native intelligence. Serving as the system orchestrator, the 6G-LLM drives a paradigm shift that fundamentally departs from existing rule-based approaches, which primarily rely on modular, experience-driven optimization. By contrast, the 6G-LLM substantially enhances network flexibility and adaptability. Nevertheless, current efforts to construct 6G-LLMs are constrained by their reliance on large-scale, meticulously curated, human-authored corpora, which are impractical to obtain in real-world scenarios. Moreover, purely offline-trained models lack the capacity for continual self-improvement, limiting their ability to adapt to the highly dynamic requirements of wireless communication environments. To overcome these limitations, we propose a novel training paradigm termed RLDTF (Reinforcement Learning from Digital Twin Feedback) for 6G-LLMs. This framework leverages network digital twins to generate reward signals based on orchestration outcomes, while employing reinforcement learning to guide the model toward optimal decision-making dynamically. Furthermore, we introduce a weighted token mechanism to improve output accuracy. Comprehensive experimental results demonstrate that our proposed framework significantly outperforms state-of-the-art baselines in orchestration accuracy and solution optimality.

Figures (8)

• Figure 1: System diagram of the proposed native intelligence communication system enabled by 6G-LLMs.
• Figure 2: The training process the proposed 6G-LLMs.
• Figure 3: Experimental framework for model training and inference.
• Figure 4: Convergence curve of the loss function during reinforcement training.
• Figure 5: Curve of the reward value during reinforcement training.