LLM-based Discriminative Reasoning for Knowledge Graph Question Answering

TL;DR

This paper addresses hallucination in LLM-based knowledge graph question answering by reframing KGQA as three discriminative subtasks: question-related subgraph searching, subgraph pruning, and answer inference. The READS framework uses discriminative strategies to guide subgraph construction and constraint handling, coupled with structured representations ($G_k$ and $S_k$) and entity-type labeling to improve grounding. Empirical results on WebQSP and CWQ show state-of-the-art performance and reduced ungrounded reasoning, with analyses demonstrating improved subgraph retrieval, pruning efficiency, and robustness across backbone models. The work advances practical KGQA by increasing reliability and efficiency, albeit assuming pre-linked entities and some dependence on gold queries as noted in its limitations.

Abstract

Large language models (LLMs) based on generative pre-trained Transformer have achieved remarkable performance on knowledge graph question-answering (KGQA) tasks. However, LLMs often produce ungrounded subgraph planning or reasoning results in KGQA due to the hallucinatory behavior brought by the generative paradigm. To tackle this issue, we propose READS to reformulate the KGQA process into discriminative subtasks, which simplifies the search space for each subtasks. Based on the subtasks, we design a new corresponding discriminative inference strategy to conduct the reasoning for KGQA, thereby alleviating hallucination and ungrounded reasoning issues in LLMs. Experimental results show that the proposed approach outperforms multiple strong comparison methods, along with achieving state-of-the-art performance on widely used benchmarks WebQSP and CWQ.

Figures (6)

• Figure 1: The generation-based methods tend to generate unsupported or redundant subgraphs and reasoning results (left), while the proposed method address the issue by establishing proper searching space for each of the KGQA subtasks (right).
• Figure 2: The proposed READS for KGQA. Start from the question (bottom left) with a given starting entity "Coronation Street", READS sequentially conducts subgraph retrieval, subgraph pruning, and answer inference. Then READS automatically uses the reasoning results to prune $G_k$ and then retrieve the answers from it. In this figure, the node's color in subgraph $G_k$ (middle center) represents its position in subgraph structure $S_k$ (top center).
• Figure 3: The number of cases with the size (number of triplets) of retrieved subgraphs.
• Figure 4: The trend of average Hits@1 as the size (number of triplets) of retrieved subgraph increases.
• Figure 5: Case frequency of different types of errors, E1 corresponds to searching subtask; E2 corresponds to pruning subtask; E3 corresponds to answering subtask.