Empowering Large Language Models in Wireless Communication: A Novel Dataset and Fine-Tuning Framework

TL;DR

The paper tackles the scarcity of domain-specific, multi-hop reasoning data for wireless LLMs by introducing a dedicated wireless dataset and a Pointwise V-Information (PVI)–based fine-tuning framework with LoRA. It defines the PVI metric as $PVI(q\\to y)=\\log_2\\frac{p(y|q)}{p(y)}$ to quantify information gained from each example and uses curriculum-like data ordering to guide training. Empirically, it reports ROUGE-L gains up to 20.9% for summarization and model-specific improvements like 2.24% and 1.31%, plus scaling-law insights for data efficiency and edge deployment. The framework enables practical domain adaptation of LLMs in wireless networks and provides a pipeline for dataset construction, multi-hop question generation, and efficient training on resource-constrained devices.

Abstract

In this work, we develop a specialized dataset aimed at enhancing the evaluation and fine-tuning of large language models (LLMs) specifically for wireless communication applications. The dataset includes a diverse set of multi-hop questions, including true/false and multiple-choice types, spanning varying difficulty levels from easy to hard. By utilizing advanced language models for entity extraction and question generation, rigorous data curation processes are employed to maintain high quality and relevance. Additionally, we introduce a Pointwise V-Information (PVI) based fine-tuning method, providing a detailed theoretical analysis and justification for its use in quantifying the information content of training data with 2.24\% and 1.31\% performance boost for different models compared to baselines, respectively. To demonstrate the effectiveness of the fine-tuned models with the proposed methodologies on practical tasks, we also consider different tasks, including summarizing optimization problems from technical papers and solving the mathematical problems related to non-orthogonal multiple access (NOMA), which are generated by using the proposed multi-agent framework. Simulation results show significant performance gain in summarization tasks with 20.9\% in the ROUGE-L metrics. We also study the scaling laws of fine-tuning LLMs and the challenges LLMs face in the field of wireless communications, offering insights into their adaptation to wireless communication tasks. This dataset and fine-tuning methodology aim to enhance the training and evaluation of LLMs, contributing to advancements in LLMs for wireless communication research and applications.

Figures (13)

• Figure 1: Structure and outline of Section \ref{['sec:III']} and Section \ref{['sec:IV']}. The methodology outlines constructing a high-quality wireless communications dataset by retrieving and sanitizing articles, automatically extracting key technical entities with LLMs, and curating coherent multi-hop reasoning examples. Using NOMA as an example, the process integrates sequential subquestions into complex queries, ensures logical consistency through reasoning chains, validates answers, and applies bias mitigation strategies to maintain accuracy and impartiality.
• Figure 2: Example question generated by using multi-agent (See Appendix D).
• Figure 3: Performance gain comparison across subset sizes for GPT-2 Large, GPT-2 XL, and LLaMA-2 7B models. While fine-tuning leads to consistent performance improvements, emphasizing its advantage on task-specific enhancements. Interestingly, the relatively straightforward questions, exemplified by the one illustrated in this figure, were evaluated across various LLMs, with even several advanced models failing to produce the correct answers, including LLaMA-3.1 8B llama3, GPT-4o-mini, etc.
• Figure 4: Comparsions of performance gains across different data ordering strategies for GPT2-large and LLaMA-2 7B.
• Figure 5: Studies on the power allocation, energy efficiency, fairness and QoS in two-user NOMA case, using LLaMA-2 7B with and without fine-tuning with $R_{min} = 2$.