Learning to Retrieve and Reason on Knowledge Graph through Active Self-Reflection

TL;DR

This work tackles knowledge graph question answering by introducing Active self-Reflection Graph Reasoning (ArG), an end-to-end framework that performs on-demand retrieval and iterative, reflective reasoning over graphs. ArG employs four self-reflection token types and a hypo-generator to continuously assess relevance, rationality, and usefulness while exploring a reasoning tree, enabling interpretable paths and more efficient retrieval. Empirical results on WebQSP and CWQ show state-of-the-art or competitive performance, with ablations confirming the importance of the reflection tokens and the hypo-generator, and transfer studies demonstrating adaptability to other KGs. The approach offers practical impact by improving reasoning transparency and retrieval efficiency in KGQA, and future work could incorporate RLHF and deeper tree-based reasoning to further enhance precision.

Abstract

Extensive research has investigated the integration of large language models (LLMs) with knowledge graphs to enhance the reasoning process. However, understanding how models perform reasoning utilizing structured graph knowledge remains underexplored. Most existing approaches rely on LLMs or retrievers to make binary judgments regarding the utilization of knowledge, which is too coarse. Meanwhile, there is still a lack of feedback mechanisms for reflection and correction throughout the entire reasoning path. This paper proposes an Active self-Reflection framework for knowledge Graph reasoning ARG, introducing for the first time an end-to-end training approach to achieve iterative reasoning grounded on structured graphs. Within the framework, the model leverages special tokens to \textit{actively} determine whether knowledge retrieval is necessary, performs \textit{reflective} critique based on the retrieved knowledge, and iteratively reasons over the knowledge graph. The reasoning paths generated by the model exhibit high interpretability, enabling deeper exploration of the model's understanding of structured knowledge. Ultimately, the proposed model achieves outstanding results compared to existing baselines in knowledge graph reasoning tasks.

Figures (8)

• Figure 1: An example of ArG performs a more fine-grained assessment and actively retrieves knowledge (relations) compared to LLM pruning and direct retrieval.
• Figure 2: The overall framework of ArG. Given an input query, the trained generator model $\mathcal{M}$ iteratively performs knowledge retrieval over the structual graph based on the retrieval token. Subsequently, the retrieved knowledge undergoes processes of critique and reflection, where implausible information is filtered. The iterative procedure culminates in the generation of an answer. ArG exhibits strong interpretability when applied to structured graph. As demonstrated in the example, the step-by-step reasoning path is organized in the lower half.
• Figure 3: Ablation results of different reasoning depth and search depth on the WebQSP and CWQ.
• Figure 4: An example of ArG training data.
• Figure 5: Instructions for [draw=myblue,thick,inner sep=2pt]test Relevance Token (for relations).