Beyond Chains: Bridging Large Language Models and Knowledge Bases in Complex Question Answering

TL;DR

This paper tackles knowledge base question answering (KBQA) with large language models by addressing the limitations of chain-only KG‑RAG methods through a training‑free four‑stage framework called PDRR (Predict–Decompose–Retrieve–Reason). PDRR introduces an explicit planning module that predicts question type (chain vs parallel) and decomposes queries into structured decomposition triples, followed by a retrieval and reasoning module that grounds and executes the plan over a knowledge graph. The approach handles both chain and non‑chain (e.g., conjunction) reasoning, showing robust improvements across multiple backbones (e.g., GPT‑4o, GPT‑3.5‑turbo) and datasets (CWQ, WebQSP, SimpleQuestions, GrailQA). Empirical results demonstrate that PDRR outperforms training‑free baselines and remains competitive with training‑based methods, highlighting the value of explicit planning and principled grounding in complex KBQA. The work advances practical KBQA by delivering transparent, scalable reasoning workflows that mitigate hallucination and knowledge staleness while preserving interpretability through decomposition triples and planning.

Abstract

Knowledge Base Question Answering (KBQA) aims to answer natural language questions using structured knowledge from KBs. While LLM-only approaches offer generalization, they suffer from outdated knowledge, hallucinations, and lack of transparency. Chain-based KG-RAG methods address these issues by incorporating external KBs, but are limited to simple chain-structured questions due to the absence of planning and logical structuring. Inspired by semantic parsing methods, we propose PDRR: a four-stage framework consisting of Predict, Decompose, Retrieve, and Reason. Our method first predicts the question type and decomposes the question into structured triples. Then retrieves relevant information from KBs and guides the LLM as an agent to reason over and complete the decomposed triples. Experimental results demonstrate that PDRR consistently outperforms existing methods across various LLM backbones and achieves superior performance on both chain-structured and non-chain complex questions.

Figures (5)

• Figure 1: Drawbacks of ToG (a chain-based KG-RAG approach). ToG and similar chain-based KG-RAG methods lack a planning module for explicit reasoning control and are limited to chain-type questions due to their insufficient logical structuring. Our PDRR framework resolves both issues.
• Figure 2: The framework of the PDRR method. The process follows the Predict-Decompose-Retrieve-Reason pipeline. Dashed lines and circles indicate pruned components with low relevance to the specific decomposed triple $\mathcal{T}_{n}^{q,D}$. Labeled entity#index (e.g., artist#1) elements denote key bridge entities $e_{n}^{B}$ that are essential for reasoning.
• Figure 3: Predicted question structure types (chain or parallel) by different LLMs on various question types in the CWQ dataset. Blue indicates that the LLM predicts the question as a chain structure, while orange indicates a parallel structure.
• Figure 4: Hits@1 accuracy of different question types in the CWQ dataset under chain and parallel reasoning strategies. The evaluation includes 500 chain, 500 intersection, 213 comparative, and 197 superlative questions. GPT-4o is used as the LLM backbone.
• Figure 5: Hits@1 accuracy on the first 500 composition-type questions in CWQ using chain reasoning with different number of retained triples.