DO-RAG: A Domain-Specific QA Framework Using Knowledge Graph-Enhanced Retrieval-Augmented Generation

TL;DR

DO-RAG tackles the challenge of ensuring factual accuracy in domain-specific QA by grounding responses in a dynamic, multi-level knowledge graph while leveraging retrieval-augmented generation. It introduces an agentic chain-of-thought extraction pipeline to construct multimodal knowledge graphs, fuses graph traversal with semantic retrieval, and employs a grounded refinement to mitigate hallucinations. In experiments on SunDB and Electrical domains, it achieves near-perfect contextual recall and over 94% answer relevancy, outperforming baselines by up to 33.38%. This work demonstrates a scalable, adaptable framework that combines structured knowledge with generative reasoning to deliver high-precision QA across domains.

Abstract

Domain-specific QA systems require not just generative fluency but high factual accuracy grounded in structured expert knowledge. While recent Retrieval-Augmented Generation (RAG) frameworks improve context recall, they struggle with integrating heterogeneous data and maintaining reasoning consistency. To address these challenges, we propose DO-RAG, a scalable and customizable hybrid QA framework that integrates multi-level knowledge graph construction with semantic vector retrieval. Our system employs a novel agentic chain-of-thought architecture to extract structured relationships from unstructured, multimodal documents, constructing dynamic knowledge graphs that enhance retrieval precision. At query time, DO-RAG fuses graph and vector retrieval results to generate context-aware responses, followed by hallucination mitigation via grounded refinement. Experimental evaluations in the database and electrical domains show near-perfect recall and over 94% answer relevancy, with DO-RAG outperforming baseline frameworks by up to 33.38%. By combining traceability, adaptability, and performance efficiency, DO-RAG offers a reliable foundation for multi-domain, high-precision QA at scale.

Figures (5)

• Figure 1: High-level overview of DO-RAG methodology.
• Figure 2: Multi-level entity-relation extraction.
• Figure 3: Overview of the retrieval process.
• Figure 4: Overall response pipeline.
• Figure 5: Composite score comparison across frameworks and language models.