Grounding LLM Reasoning with Knowledge Graphs

TL;DR

We address the verifiability gap in LLM outputs by grounding stepwise reasoning in domain-specific knowledge graphs. The authors propose a framework that binds each reasoning step to graph data and evaluate three reasoning strategies—CoT, ToT, and GoT—coupled with two KG interaction pipelines (Agent and Automatic Graph Exploration). The approach delivers state-of-the-art performance on GRBench (≥26.5% improvement over CoT) and provides detailed analyses of how step depth, branching, and model size shape reasoning quality. While grounding improves interpretability and reduces certain failures, it also introduces computational costs and complexities, especially for ToT and GoT, highlighting practical trade-offs for scalable, verifiable KG-grounded reasoning in real-world settings.

Abstract

Large Language Models (LLMs) excel at generating natural language answers, yet their outputs often remain unverifiable and difficult to trace. Knowledge Graphs (KGs) offer a complementary strength by representing entities and their relationships in structured form, providing a foundation for more reliable reasoning. We propose a novel framework that integrates LLM reasoning with KGs by linking each step of the reasoning process to graph-structured data. This grounding turns intermediate ``thoughts'' into interpretable traces that remain consistent with external knowledge. Our approach incorporates multiple reasoning strategies, Chain-of-Thought (CoT), Tree-of-Thought (ToT), and Graph-of-Thought (GoT), and is evaluated on GRBench, a benchmark for domain-specific graph reasoning. Our experiments show state-of-the-art (SOTA) performance, with at least 26.5\% improvement over CoT baselines. Beyond accuracy, we analyze how step depth, branching structure, and model size influence reasoning quality, offering insights into the conditions that support effective reasoning. Together, these contributions highlight how grounding LLMs in structured knowledge enables both higher accuracy and greater interpretability in complex reasoning tasks.

Figures (15)

• Figure 1: Methods for Question-Answering in KGs (Section \ref{['Section:Method']}). Left: Agent. LLM decides to take one of the predefined actions to connect with the graph. Right:Automatic Graph Exploration. Entities are extracted in each reasoning step, triggering a search for each identified entity.
• Figure 2: Reasoning Strategies: This figure illustrates different LLM reasoning strategies to navigate the potential answer space: CoT, ToT, GoT. Each strategy consists of "thoughts" connected to the KG through search methods (Section \ref{['Section:LM-KG-Interaction']}) illustrating stepwise reasoning over structured knowledge.
• Figure 3: Automatic Graph Exploration. It extracts entities from text (query/thought), then select relevant relations and neighbors with the LLM. The resulting entity-relation-entity combinations form triples to expand the reasoning chain.
• Figure 4: Effect of the number of steps in the LLM-KG Interaction Methods. The Agent requires more steps to obtain the performance of the Graph Exploration, while the Graph Exploration only needs the anchor entities to perform the search within the graph.
• Figure 5: Effect of the Search depth in Graph Exploration interaction method for a fixed steps number. The method can achieve relatively good performance with the anchor entities extracted from the question.