Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Telecom Specific LLM: TSLAM-Mini with QLoRA and Digital Twin Data

Vignesh Ethiraj, Divya Vijay, Sidhanth Menon, Heblin Berscilla

TL;DR

TSLAM-Mini addresses the gap between general-purpose LLMs and real-time telecom needs by fine-tuning a compact 3.8B model on a 100k-sample, 20-use-case telecom dataset synthesized via DigiTwin simulations, RFC ingestion, and SME input. Leveraging Quantized Low-Rank Adaptation (QLoRA) enables efficient training, with an automated Qwen3-235B-A22B adjudicator providing objective evaluation of instruction-following and telecom relevance. The work demonstrates that domain-specific data and PEFT can yield strong, edge-friendly performance superior to larger generalist models on telecom tasks. This approach offers a practical blueprint for deploying compact, domain-expert LLMs in real-time network management and operations.

Abstract

General-purpose large language models (LLMs), despite their broad capabilities accrued from open-world data, frequently exhibit suboptimal performance when confronted with the nuanced and specialized demands inherent in real-time telecommunications applications. This investigation addresses this critical limitation through the meticulous fine-tuning of TSLAM-Mini developed by NetoAI, a compact (3.8-billion parameter) causal language model architecturally derived from Phi-4 Mini Instruct 4B. The fine-tuning regimen leverages a bespoke dataset comprising 100,000 samples, strategically engineered to address 20 pivotal telecommunications use-cases, encompassing domains such as Network Fundamentals, IP Routing, MPLS, Network Security, Automation, OSS/BSS, RAN, Mobile Core, Satellite Communications, and Ethical AI. This dataset was curated utilizing NetoAI's DigiTwin platform, enriched with granular insights from venerated network Subject Matter Experts (SMEs) and authoritative RFC documents, thereby capturing high-fidelity representations of real-world network dynamics through simulations inspired by digital twin paradigms. Employing Quantized Low-Rank Adaptation (QLoRA), a state-of-the-art Parameter Efficient Fine-Tuning (PEFT) technique, we achieved substantial training efficiency and enabled prospective deployment on resource-constrained hardware. A novel evaluation framework, predicated on a high-capacity LLM (Qwen3-235B-A22B) functioning as an automated adjudicator, was instituted to rigorously assess instruction-following fidelity and response quality across the specified telecom use-cases. Empirical results unequivocally demonstrate TSLAM-Mini's superior aptitude in telecom-centric applications, underscoring the profound efficacy of domain-specific datasets and PEFT methodologies for advancing intelligent network management.

Efficient Telecom Specific LLM: TSLAM-Mini with QLoRA and Digital Twin Data

TL;DR

TSLAM-Mini addresses the gap between general-purpose LLMs and real-time telecom needs by fine-tuning a compact 3.8B model on a 100k-sample, 20-use-case telecom dataset synthesized via DigiTwin simulations, RFC ingestion, and SME input. Leveraging Quantized Low-Rank Adaptation (QLoRA) enables efficient training, with an automated Qwen3-235B-A22B adjudicator providing objective evaluation of instruction-following and telecom relevance. The work demonstrates that domain-specific data and PEFT can yield strong, edge-friendly performance superior to larger generalist models on telecom tasks. This approach offers a practical blueprint for deploying compact, domain-expert LLMs in real-time network management and operations.

Abstract

General-purpose large language models (LLMs), despite their broad capabilities accrued from open-world data, frequently exhibit suboptimal performance when confronted with the nuanced and specialized demands inherent in real-time telecommunications applications. This investigation addresses this critical limitation through the meticulous fine-tuning of TSLAM-Mini developed by NetoAI, a compact (3.8-billion parameter) causal language model architecturally derived from Phi-4 Mini Instruct 4B. The fine-tuning regimen leverages a bespoke dataset comprising 100,000 samples, strategically engineered to address 20 pivotal telecommunications use-cases, encompassing domains such as Network Fundamentals, IP Routing, MPLS, Network Security, Automation, OSS/BSS, RAN, Mobile Core, Satellite Communications, and Ethical AI. This dataset was curated utilizing NetoAI's DigiTwin platform, enriched with granular insights from venerated network Subject Matter Experts (SMEs) and authoritative RFC documents, thereby capturing high-fidelity representations of real-world network dynamics through simulations inspired by digital twin paradigms. Employing Quantized Low-Rank Adaptation (QLoRA), a state-of-the-art Parameter Efficient Fine-Tuning (PEFT) technique, we achieved substantial training efficiency and enabled prospective deployment on resource-constrained hardware. A novel evaluation framework, predicated on a high-capacity LLM (Qwen3-235B-A22B) functioning as an automated adjudicator, was instituted to rigorously assess instruction-following fidelity and response quality across the specified telecom use-cases. Empirical results unequivocally demonstrate TSLAM-Mini's superior aptitude in telecom-centric applications, underscoring the profound efficacy of domain-specific datasets and PEFT methodologies for advancing intelligent network management.
Paper Structure (20 sections, 1 equation, 5 figures, 1 table, 1 algorithm)

This paper contains 20 sections, 1 equation, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Parameter-Efficient Fine-Tuning (PEFT) for LLMs
  4. Domain-Specific Fine-Tuning of LLMs
  5. Fine-Tuning Phi-4 Mini Instruct 4B
  6. Synthetic Data Generation and Digital Twins in Telecom
  7. Methodology
  8. Dataset Creation and Curation
  9. Fine-Tuning with Quantized Low-Rank Adaptation (QLoRA)
  10. Fine-Tuning Configuration and Targeted Layers
  11. Loss Function
  12. Optimization Techniques
  13. Evaluation Framework
  14. Judge Model Selection
  15. Automated Evaluation Pipeline
  16. ...and 5 more sections

Figures (5)

  • Figure 1: High-level architecture diagram illustrating the TSLAM-Mini development and evaluation pipeline, including data sources, fine-tuning process, and evaluation framework.
  • Figure 2: Training loss progression for TSLAM-Mini, illustrating convergence from an initial loss of 2.6418 to 0.1511 over 124.5 million tokens.
  • Figure 3: Benchmark heatmap illustrating comparative performance (%) of TSLAM-Mini and other language models across various standardized tasks.
  • Figure 4: Comparative performance of TSLAM-Mini against selected larger general-purpose LLMs across diverse evaluation benchmarks. Despite having only 2.28B parameters after quantization, TSLAM-Mini demonstrates strong domain-specific and general capabilities across multiple task categories.
  • Figure 5: Comparative performance of TSLAM-Mini against selected larger general-purpose LLMs on telecom-specific evaluation benchmarks. TSLAM-Mini demonstrates strong performance on telecom domain tasks.