Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bisecting K-Means in RAG for Enhancing Question-Answering Tasks Performance in Telecommunications

Pedro Sousa, Cláudio Klautau Mello, Frank B. Morte, Luis F. Solis Navarro

TL;DR

This work tackles question-answering in the telecommunications domain by integrating a routing module with Bisecting K-Means clustering into a Retrieval-Augmented Generation pipeline. It processes 3GPP-release documents into content-based clusters, enabling efficient subset retrieval via a vector database and a top-K routing strategy, while fine-tuning a small phi-2 model on RAG-context data. Empirical results show accuracy improvements of 66.12% on phi-2 and 72.13% on phi-3 with reduced training times, and the approach maintains potential applicability to larger LLMs. The work contributes a practical, scalable framework for domain-specific QA in fast-moving standards ecosystems.

Abstract

Question-answering tasks in the telecom domain are still reasonably unexplored in the literature, primarily due to the field's rapid changes and evolving standards. This work presents a novel Retrieval-Augmented Generation framework explicitly designed for the telecommunication domain, focusing on datasets composed of 3GPP documents. The framework introduces the use of the Bisecting K-Means clustering technique to organize the embedding vectors by contents, facilitating more efficient information retrieval. By leveraging this clustering technique, the system pre-selects a subset of clusters that are most similar to the user's query, enhancing the relevance of the retrieved information. Aiming for models with lower computational cost for inference, the framework was tested using Small Language Models, demonstrating improved performance with an accuracy of 66.12% on phi-2 and 72.13% on phi-3 fine-tuned models, and reduced training time.

Bisecting K-Means in RAG for Enhancing Question-Answering Tasks Performance in Telecommunications

TL;DR

This work tackles question-answering in the telecommunications domain by integrating a routing module with Bisecting K-Means clustering into a Retrieval-Augmented Generation pipeline. It processes 3GPP-release documents into content-based clusters, enabling efficient subset retrieval via a vector database and a top-K routing strategy, while fine-tuning a small phi-2 model on RAG-context data. Empirical results show accuracy improvements of 66.12% on phi-2 and 72.13% on phi-3 with reduced training times, and the approach maintains potential applicability to larger LLMs. The work contributes a practical, scalable framework for domain-specific QA in fast-moving standards ecosystems.

Abstract

Question-answering tasks in the telecom domain are still reasonably unexplored in the literature, primarily due to the field's rapid changes and evolving standards. This work presents a novel Retrieval-Augmented Generation framework explicitly designed for the telecommunication domain, focusing on datasets composed of 3GPP documents. The framework introduces the use of the Bisecting K-Means clustering technique to organize the embedding vectors by contents, facilitating more efficient information retrieval. By leveraging this clustering technique, the system pre-selects a subset of clusters that are most similar to the user's query, enhancing the relevance of the retrieved information. Aiming for models with lower computational cost for inference, the framework was tested using Small Language Models, demonstrating improved performance with an accuracy of 66.12% on phi-2 and 72.13% on phi-3 fine-tuned models, and reduced training time.

Paper Structure

This paper contains 11 sections, 2 equations, 8 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Query Enhancement
  5. Preprocessing of Document Texts
  6. Clustering Methods for RAG
  7. Routing Module
  8. Fine-tuned Model
  9. Results
  10. Future Works
  11. Conclusions

Figures (8)

  • Figure 1: Diagram of the RAG pipeline that integrates a routing module with clustering techniques to efficiently retrieve the most relevant documents.
  • Figure 2: The process of query enhancement before retrieving information and passing it to the LLM for generating an answer.
  • Figure 3: Performance of phi-2 on different chunk sizes.
  • Figure 4: Performance difference between regular K-Means and Bisecting K-Means clustering, all performed using the phi-2 model.
  • Figure 5: Routing module.
  • ...and 3 more figures