Search-on-Graph: Iterative Informed Navigation for Large Language Model Reasoning on Knowledge Graphs

TL;DR

The paper addresses the reliability gap of large language models in knowledge-intensive, multi-hop reasoning by introducing Search-on-Graph (SoG), a simple observation-driven KGQA framework that relies on a single 1-hop Search function to iteratively navigate knowledge graphs. SoG foregoes upfront path planning and large subgraph retrieval, instead having the LLM observe available relations at each hop and make informed decisions, with adaptive filtering for high-degree nodes and carefully engineered prompts. Across six KGQA benchmarks on Freebase and Wikidata, SoG achieves state-of-the-art results without fine-tuning, with particularly strong gains on Wikidata benchmarks (approximately +16 percentage points). The contributions include a compact, generalizable navigation mechanism, analysis of design choices (output formats, few-shot exemplars, model variants), and reproducible prompts and tooling, illustrating that observation-centric design can unlock robust, schema-agnostic reasoning in LLMs for KGQA.

Abstract

Large language models (LLMs) have demonstrated impressive reasoning abilities yet remain unreliable on knowledge-intensive, multi-hop questions -- they miss long-tail facts, hallucinate when uncertain, and their internal knowledge lags behind real-world change. Knowledge graphs (KGs) offer a structured source of relational evidence, but existing KGQA methods face fundamental trade-offs: compiling complete SPARQL queries without knowing available relations proves brittle, retrieving large subgraphs introduces noise, and complex agent frameworks with parallel exploration exponentially expand search spaces. To address these limitations, we propose Search-on-Graph (SoG), a simple yet effective framework that enables LLMs to perform iterative informed graph navigation using a single, carefully designed \textsc{Search} function. Rather than pre-planning paths or retrieving large subgraphs, SoG follows an ``observe-then-navigate'' principle: at each step, the LLM examines actual available relations from the current entity before deciding on the next hop. This approach further adapts seamlessly to different KG schemas and handles high-degree nodes through adaptive filtering. Across six KGQA benchmarks spanning Freebase and Wikidata, SoG achieves state-of-the-art performance without fine-tuning. We demonstrate particularly strong gains on Wikidata benchmarks (+16\% improvement over previous best methods) alongside consistent improvements on Freebase benchmarks.

Figures (2)

• Figure 1: SoG workflow for the query "What is the capital of Vincent van Gogh's birth country?" The LLM iteratively navigates the KG, with reasoning and Search calls shown on the left and KG navigation on the right. The path follows Van Gogh $\rightarrow$ Zundert (place of birth) $\rightarrow$ Netherlands (contained by) $\rightarrow$ Amsterdam (capital). Solid boxes indicate selected entities; dotted boxes show unselected retrieved entities. Tables display the markdown output returned by Search, revealing available 1-hop neighbours from each entity.
• Figure 2: Impact of few-shot exemplar quantity on exact match accuracy (%) across four Freebase datasets (SimpleQA, WebQSP, CWQ, GrailQA) for Qwen3-30B-A3B-Thinking-2507 and Qwen3-30B-A3B-Instruct-2507.