Dual Reasoning: A GNN-LLM Collaborative Framework for Knowledge Graph Question Answering

TL;DR

This work introduces Dual-Reasoning (DualR), a two-tier framework for KGQA that couples an external GNN-based System 2 with an LLM-based System 1. An LLM-empowered GNN explicitly reasons over knowledge graphs to extract high-quality reasoning chains, which are refined into a knowledge-enhanced multiple-choice prompt guiding a frozen LLM to produce final answers. The approach achieves state-of-the-art performance on WebQSP, CWQ, and MetaQA while maintaining high efficiency via one-step LLM inference and interpretable reasoning. Pretraining the GNN on multiple KGQA datasets further improves generalization, and ablations confirm the value of each component, including the prompt design. Overall, DualR demonstrates the benefit of integrating structured graph learning with powerful language understanding for faithful and efficient KGQA.

Abstract

Large Language Models (LLMs) excel at intuitive, implicit reasoning. Guiding LLMs to construct thought chains can enhance their deliberate reasoning abilities, but also faces challenges such as hallucination. Knowledge Graphs (KGs) can provide explicit structured knowledge for LLMs to alleviate these issues. However, existing KG-enhanced methods often overlook explicit graph learning, making it challenging to efficiently provide precise reasoning chains for LLMs. Following dual-process theory, we propose Dual-Reasoning (DualR), a novel framework that integrates an external system based on Graph Neural Network (GNN) for explicit reasoning on KGs, complementing the implicit reasoning of LLMs through externalized reasoning chains. DualR designs an LLM-empowered GNN module for explicit learning on KGs, efficiently extracting high-quality reasoning chains. These reasoning chains are then refined to a knowledge-enhanced multiple-choice prompt, guiding a frozen LLM to reason thoughtfully for final answer determination. Extensive experiments on three benchmark KGQA datasets demonstrate that DualR achieves state-of-the-art performance while maintaining high efficiency and interpretability.

Figures (6)

• Figure 1: Illustration of the Dual-Reasoning (DualR), a GNN-LLM collaborative framework for knowledge graph question answering. It contains two-tier reasoning: (1) semantic-aware knowledge exploration; and (2) knowledge-enhanced answer determination.
• Figure 2: Illustration of semantic-aware knowledge exploration. We start exploration from topic entity Birdy. In each step, we firstly get the unpruned candidate set $\mathcal{C}_q^\ell$, calculate the attention weights $\alpha_{q|sr}^\ell$ of different edges, prune several irrelevant entities (in white), and update candidate set $\widetilde{\mathcal{C}}_q^\ell$ (in blue). The representations are propagated from entities in $\widetilde{\mathcal{C}}_q^{\ell-1}$ to $\widetilde{\mathcal{C}}_q^\ell$ through an one-layer GNN.
• Figure 3: Examples of accurate and faithful output by DualR-Llama2-13B.
• Figure 4: Influence of pruning on three datasets.
• Figure 5: Learning curves on two datasets.