Telco-oRAG: Optimizing Retrieval-augmented Generation for Telecom Queries via Hybrid Retrieval and Neural Routing

TL;DR

Telco-oRAG tackles the brittleness of LLMs in telecom by integrating a hybrid retrieval strategy that combines 3GPP-standard content with open-web data, guided by glossary-enhanced query refinement and a neural router to curb memory usage. It introduces dual-round 3GPP retrieval and structured prompting to ground responses, achieving significant accuracy gains (up to 17.6% in MCQs and 10.6% in lexicon questions) and a 45% RAM reduction. The framework demonstrates robustness across multiple LLMs, including open-source models, and enables near GPT-4-level performance on telecom benchmarks while enabling deployment on memory-constrained devices. The authors release Telco-oRAG as open-source, extendable to other standards, and provide a comprehensive latency and memory-footprint analysis to guide practical deployment.

Abstract

Artificial intelligence will be one of the key pillars of the next generation of mobile networks (6G), as it is expected to provide novel added-value services and improve network performance. In this context, large language models have the potential to revolutionize the telecom landscape through intent comprehension, intelligent knowledge retrieval, coding proficiency, and cross-domain orchestration capabilities. This paper presents Telco-oRAG, an open-source Retrieval-Augmented Generation (RAG) framework optimized for answering technical questions in the telecommunications domain, with a particular focus on 3GPP standards. Telco-oRAG introduces a hybrid retrieval strategy that combines 3GPP domain-specific retrieval with web search, supported by glossary-enhanced query refinement and a neural router for memory-efficient retrieval. Our results show that Telco-oRAG improves the accuracy in answering 3GPP-related questions by up to 17.6% and achieves a 10.6% improvement in lexicon queries compared to baselines. Furthermore, Telco-oRAG reduces memory usage by 45% through targeted retrieval of relevant 3GPP series compared to baseline RAG, and enables open-source LLMs to reach GPT-4-level accuracy on telecom benchmarks.

Figures (13)

• Figure 1: The proposed Telco-oRAG pipeline. The left side shows the web search block; the right side shows the content retrieval from the 3GPP database.
• Figure 2: t-SNE projection of the embeddings of 3GPP documents and user query after the proposed processing stages.
• Figure 3: The proposed NN router architecture.
• Figure 4: Telco-oRAG frontend. Left side: the answer provided by Telco-oRAG to the user query. Right side: the selected retrievals that support the generation of the answer.
• Figure 5: RAG accuracy vs. context length.