KG2QA: Knowledge Graph-enhanced Retrieval-augmented Generation for Communication Standards Question Answering

TL;DR

This work tackles the inefficiency of consulting a growing landscape of communication standards by introducing KG2QA, a retrieval-augmented QA system that fuses a domain-specific knowledge graph with fine-tuned large language models. It contributes a 6,587 QA-pair dataset derived from ITU-T recommendations, a structured KG with 13,906 entities and 13,524 relations built via LLM-assisted extraction, and a KG-RAG QA pipeline where KG context informs generation. The framework is openly available, with Qwen-2.5-7B-Instruct fine-tuning via LoRA, Neo4j-based KG storage, and an end-to-end Web/API deployment, achieving improved factual grounding as judged by DeepSeek-V3 and a 2.26% average improvement for the Qwen-Merged variant. The approach enhances query accuracy, speed, and user experience, and can be extended to other standardization domains.

Abstract

The rapid evolution of communication technologies has led to an explosion of standards, rendering traditional expert-dependent consultation methods inefficient and slow. To address this challenge, we propose \textbf{KG2QA}, a question answering (QA) framework for communication standards that integrates fine-tuned large language models (LLMs) with a domain-specific knowledge graph (KG) via a retrieval-augmented generation (RAG) pipeline. We construct a high-quality dataset of 6,587 QA pairs from ITU-T recommendations and fine-tune Qwen2.5-7B-Instruct, achieving significant performance gains: BLEU-4 increases from 18.86 to 66.90, outperforming both the base model and Llama-3-8B-Instruct. A structured KG containing 13,906 entities and 13,524 relations is built using LLM-assisted triple extraction based on a custom ontology. In our KG-RAG pipeline, the fine-tuned LLMs first retrieves relevant knowledge from KG, enabling more accurate and factually grounded responses. Evaluated by DeepSeek-V3 as a judge, the KG-enhanced system improves performance across five dimensions, with an average score increase of 2.26\%, demonstrating superior factual accuracy and relevance. Integrated with Web platform and API, KG2QA delivers an efficient and interactive user experience. Our code and data have been open-sourced https://github.com/luozhongze/KG2QA.

Figures (4)

• Figure 1: The overall architecture of KG2QA.
• Figure 2: Prompt of LLM-based judge.
• Figure 3: The KG constructed.
• Figure 4: The multi-round QA example of the Web platform.