A Large Language Model Based Method for Complex Logical Reasoning over Knowledge Graphs

TL;DR

This work tackles the challenge of answering complex first-order logic queries over incomplete knowledge graphs by introducing ROG, an ensemble framework that combines query-aware subgraph retrieval with large language model (LLM) chain-of-thought reasoning. By decomposing complex queries into simple sub-queries and reasoning step-by-step within query-relevant subgraphs, ROG avoids heavy task-specific embeddings and leverages LLM inference to handle deep reasoning chains. Empirical results on FB15k and NELL995 show substantial mean reciprocal rank improvements (approximately 35%-55%) over strong embedding-based baselines, especially for high-complexity query types. The approach demonstrates that integrating structured KG retrieval with LLM-driven reasoning offers a flexible, robust alternative for complex knowledge-graph reasoning in noisy, large-scale settings.

Abstract

Reasoning over knowledge graphs (KGs) with first-order logic (FOL) queries is challenging due to the inherent incompleteness of real-world KGs and the compositional complexity of logical query structures. Most existing methods rely on embedding entities and relations into continuous geometric spaces and answer queries via differentiable set operations. While effective for simple query patterns, these approaches often struggle to generalize to complex queries involving multiple operators, deeper reasoning chains, or heterogeneous KG schemas. We propose ROG (Reasoning Over knowledge Graphs with large language models), an ensemble-style framework that combines query-aware KG neighborhood retrieval with large language model (LLM)-based chain-of-thought reasoning. ROG decomposes complex FOL queries into sequences of simpler sub-queries, retrieves compact, query-relevant subgraphs as contextual evidence, and performs step-by-step logical inference using an LLM, avoiding the need for task-specific embedding optimization. Experiments on standard KG reasoning benchmarks demonstrate that ROG consistently outperforms strong embedding-based baselines in terms of mean reciprocal rank (MRR), with particularly notable gains on high-complexity query types. These results suggest that integrating structured KG retrieval with LLM-driven logical reasoning offers a robust and effective alternative for complex KG reasoning tasks.

Figures (2)

• Figure 1: Workflow of Logical Reasoning Decomposition and LLM-Based Answer Generation.
• Figure 2: Data flow of the ROG model.