GRASP: Generic Reasoning And SPARQL Generation across Knowledge Graphs
Sebastian Walter, Hannah Bast
TL;DR
GRASP introduces a zero-shot SPARQL QA framework that enables a large language model to generate SPARQL queries over arbitrary knowledge graphs by interactively exploring graphs through a fixed, graph-agnostic set of functions. It relies on pre-computed search indices, including a prefix-keyword and a FAISS-based similarity index, to efficiently locate relevant IRIs and literals, without any graph fine-tuning. Across benchmarks and KG domains, GRASP achieves state-of-the-art zero-shot results on Wikidata, competes closely with few-shot methods on Freebase, and demonstrates strong performance on additional graphs, with extensive ablations revealing that context-sensitive search and feedback loops improve robustness. The work emphasizes reproducibility and cross-graph applicability, offering a practical, scalable approach for SPARQL query generation in real-world KGQA scenarios.
Abstract
We propose a new approach for generating SPARQL queries on RDF knowledge graphs from natural language questions or keyword queries, using a large language model. Our approach does not require fine-tuning. Instead, it uses the language model to explore the knowledge graph by strategically executing SPARQL queries and searching for relevant IRIs and literals. We evaluate our approach on a variety of benchmarks (for knowledge graphs of different kinds and sizes) and language models (of different scales and types, commercial as well as open-source) and compare it with existing approaches. On Wikidata we reach state-of-the-art results on multiple benchmarks, despite the zero-shot setting. On Freebase we come close to the best few-shot methods. On other, less commonly evaluated knowledge graphs and benchmarks our approach also performs well overall. We conduct several additional studies, like comparing different ways of searching the graphs, incorporating a feedback mechanism, or making use of few-shot examples.