Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RouterKGQA: Specialized--General Model Routing for Constraint-Aware Knowledge Graph Question Answering

Bo Yuan, Hexuan Deng, Xuebo Liu, Min Zhang

Abstract

Knowledge graph question answering (KGQA) is a promising approach for mitigating LLM hallucination by grounding reasoning in structured and verifiable knowledge graphs. Existing approaches fall into two paradigms: retrieval-based methods utilize small specialized models, which are efficient but often produce unreachable paths and miss implicit constraints, while agent-based methods utilize large general models, which achieve stronger structural grounding at substantially higher cost. We propose RouterKGQA, a framework for specialized--general model collaboration, in which a specialized model generates reasoning paths and a general model performs KG-guided repair only when needed, improving performance at minimal cost. We further equip the specialized with constraint-aware answer filtering, which reduces redundant answers. In addition, we design a more efficient general agent workflow, further lowering inference cost. Experimental results show that RouterKGQA outperforms the previous best by 3.57 points in F1 and 0.49 points in Hits@1 on average across benchmarks, while requiring only 1.15 average LLM calls per question. Codes and models are available at https://github.com/Oldcircle/RouterKGQA.

RouterKGQA: Specialized--General Model Routing for Constraint-Aware Knowledge Graph Question Answering

Abstract

Knowledge graph question answering (KGQA) is a promising approach for mitigating LLM hallucination by grounding reasoning in structured and verifiable knowledge graphs. Existing approaches fall into two paradigms: retrieval-based methods utilize small specialized models, which are efficient but often produce unreachable paths and miss implicit constraints, while agent-based methods utilize large general models, which achieve stronger structural grounding at substantially higher cost. We propose RouterKGQA, a framework for specialized--general model collaboration, in which a specialized model generates reasoning paths and a general model performs KG-guided repair only when needed, improving performance at minimal cost. We further equip the specialized with constraint-aware answer filtering, which reduces redundant answers. In addition, we design a more efficient general agent workflow, further lowering inference cost. Experimental results show that RouterKGQA outperforms the previous best by 3.57 points in F1 and 0.49 points in Hits@1 on average across benchmarks, while requiring only 1.15 average LLM calls per question. Codes and models are available at https://github.com/Oldcircle/RouterKGQA.
Paper Structure (59 sections, 12 equations, 2 figures, 10 tables)

This paper contains 59 sections, 12 equations, 2 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Factual Hallucination of LLMs.
  4. Agent-based KGQA.
  5. Retrieval-based KGQA.
  6. Method
  7. Problem Formulation
  8. Constraint-Aware Path Generation with Specialized LLMs
  9. Initial entity.
  10. Main path.
  11. Constraint set.
  12. Specialized Model Training.
  13. Reachability verification.
  14. Efficient Path Repair with General LLMs
  15. Reasoning blueprint generation.
  16. ...and 44 more sections

Figures (2)

  • Figure 1: Overview of the RouterKGQA framework.
  • Figure 2: Sub-question consistency analysis on CWQ. (a) Distribution of sub-question set relationship categories across methods. (b) Hits@1 (solid) and F1 (dotted) by sub-question relationship category.