Enhancing Large Language Models (LLMs) for Telecommunications using Knowledge Graphs and Retrieval-Augmented Generation

TL;DR

This work tackles the gap between general-purpose LLMs and the demand for domain-specific telecom knowledge. It introduces a KG-RAG framework that grounds LLM responses in a dedicated telecom knowledge graph while leveraging retrieval to fetch the most relevant, up-to-date information during generation. The approach combines structured knowledge representations, LLM-aided entity extraction, and link prediction to maintain and expand the KG, and demonstrates superior performance over LLM-only and RAG baselines on telecom-specific summarization and QA tasks, achieving 88% accuracy on Tspec-LLM and improved ROUGE scores. The results suggest that integrating KGs with RAG can significantly enhance domain adaptation and accuracy for complex telecom queries, with potential applications in troubleshooting, standards development, and network management; future work includes real-time KG updates and broader comparisons with alternative RAG methods.

Abstract

Large language models (LLMs) have made significant progress in general-purpose natural language processing tasks. However, LLMs are still facing challenges when applied to domain-specific areas like telecommunications, which demands specialized expertise and adaptability to evolving standards. This paper presents a novel framework that combines knowledge graph (KG) and retrieval-augmented generation (RAG) techniques to enhance LLM performance in the telecom domain. The framework leverages a KG to capture structured, domain-specific information about network protocols, standards, and other telecom-related entities, comprehensively representing their relationships. By integrating KG with RAG, LLMs can dynamically access and utilize the most relevant and up-to-date knowledge during response generation. This hybrid approach bridges the gap between structured knowledge representation and the generative capabilities of LLMs, significantly enhancing accuracy, adaptability, and domain-specific comprehension. Our results demonstrate the effectiveness of the KG-RAG framework in addressing complex technical queries with precision. The proposed KG-RAG model attained an accuracy of 88% for question answering tasks on a frequently used telecom-specific dataset, compared to 82% for the RAG-only and 48% for the LLM-only approaches.

Figures (3)

• Figure 1: Illustration of the KG-RAG pipeline compared to the traditional RAG pipeline. Unlike the common RAG pipeline, the KG-RAG approach leverages a KG as a comprehensive knowledge base, providing a more structured and dynamic response capability.
• Figure 2: Accuarcies of different models by question difficulty on Tspec-LLMnikbakht2024tspec
• Figure 3: Accuarcies of different models by question difficulty on ORAN-Bench-13Kgajjar2024oran