Augmenting Question Answering with A Hybrid RAG Approach
Tianyi Yang, Nashrah Haque, Vaishnave Jonnalagadda, Yuya Jeremy Ong, Zhehui Chen, Yanzhao Wu, Lei Yu, Divyesh Jadav, Wenqi Wei
TL;DR
The paper tackles hallucination and retrieval gaps in open-domain QA by proposing Structured-Semantic RAG (SSRAG), a hybrid framework that fuses graph- and vector-based retrieval with dynamic query augmentation and an agentic routing module. SSRAG refines queries to form $Q_{ ext{aug}}$, directs routing to the most relevant sources using a deterministic TEMPORAL/FACTUAL decision, and unifies heterogeneous contexts into a single, coherent evidence set for generation. Across TruthfulQA, SQuAD, and WikiQA and five LLMs, SSRAG yields substantial improvements in factual accuracy, context relevance, and reduction of hallucinations, outperforming standard RAG and graph-only approaches. The approach is model-agnostic, scalable, and auditable, offering a practical path toward more trustworthy LLM-based QA in real-world settings.
Abstract
Retrieval-Augmented Generation (RAG) has emerged as a powerful technique for enhancing the quality of responses in Question-Answering (QA) tasks. However, existing approaches often struggle with retrieving contextually relevant information, leading to incomplete or suboptimal answers. In this paper, we introduce Structured-Semantic RAG (SSRAG), a hybrid architecture that enhances QA quality by integrating query augmentation, agentic routing, and a structured retrieval mechanism combining vector and graph based techniques with context unification. By refining retrieval processes and improving contextual grounding, our approach improves both answer accuracy and informativeness. We conduct extensive evaluations on three popular QA datasets, TruthfulQA, SQuAD and WikiQA, across five Large Language Models (LLMs), demonstrating that our proposed approach consistently improves response quality over standard RAG implementations.