Learning to Route: A Rule-Driven Agent Framework for Hybrid-Source Retrieval-Augmented Generation

TL;DR

The paper addresses domain-specific QA challenges for LLMs by integrating relational databases with unstructured documents through a rule-driven routing framework. It introduces a Rule-Driven Routing Agent, a Rule-Making Expert Agent, and a Path-Level Meta-Cache to route queries to DB, Doc, Hybrid, or direct LLM pathways, aiming to balance accuracy, latency, and cost. Empirical results on TATQA, FinQA, and WikiQA show the approach outperforms static strategies and other routing baselines, demonstrating the value of interpretable rules and adaptive rule refinement. The proposed method offers practical benefits for enterprise and scientific QA by delivering higher accuracy with moderate computational overhead and low routing latency, while maintaining interpretability and adaptability across domains.

Abstract

Large Language Models (LLMs) have shown remarkable performance on general Question Answering (QA), yet they often struggle in domain-specific scenarios where accurate and up-to-date information is required. Retrieval-Augmented Generation (RAG) addresses this limitation by enriching LLMs with external knowledge, but existing systems primarily rely on unstructured documents, while largely overlooking relational databases, which provide precise, timely, and efficiently queryable factual information, serving as indispensable infrastructure in domains such as finance, healthcare, and scientific research. Motivated by this gap, we conduct a systematic analysis that reveals three central observations: (i) databases and documents offer complementary strengths across queries, (ii) naively combining both sources introduces noise and cost without consistent accuracy gains, and (iii) selecting the most suitable source for each query is crucial to balance effectiveness and efficiency. We further observe that query types show consistent regularities in their alignment with retrieval paths, suggesting that routing decisions can be effectively guided by systematic rules that capture these patterns. Building on these insights, we propose a rule-driven routing framework. A routing agent scores candidate augmentation paths based on explicit rules and selects the most suitable one; a rule-making expert agent refines the rules over time using QA feedback to maintain adaptability; and a path-level meta-cache reuses past routing decisions for semantically similar queries to reduce latency and cost. Experiments on three QA benchmarks demonstrate that our framework consistently outperforms static strategies and learned routing baselines, achieving higher accuracy while maintaining moderate computational cost.

Figures (12)

• Figure 1: Problem Analysis
• Figure 2: Overall workflow of our rule-driven routing framework. At inference, each question first checks the Path-Level Meta-Cache; if a hit occurs, the cached path scores are reused, otherwise the query is passed to the Rule-Driven Routing Agent, which scores candidate paths based on explicit rules and updates the cache. The highest-scoring path (Database, Document, Hybrid, or Direct Answer) is then selected to provide evidence to the Summarize Agent for answer generation. During training, question–answer performance is collected and analyzed, and the Rule-Making Expert Agent refines the rule set, enabling subsequent routing decisions to adaptively improve.
• Figure 3: Accuracy and Efficiency Analysis
• Figure 4: Investigation of Proposed Method
• Figure 5: Investigation of Path Selection. For better visualization, the Basic and Doc results are proportionally scaled up while preserving their relative ratios.