Konstruktor: A Strong Baseline for Simple Knowledge Graph Question Answering

TL;DR

Konstruktor targets simple, one-hop KGQA by decomposing the problem into entity extraction/linking, relation detection, and SPARQL query generation, integrating language models with a knowledge graph for accuracy and interpretability. It evaluates multiple EL and relation-detection strategies (classification, ranking, generation) and shows that ensembles combining these approaches often outperform end-to-end LLMs on several Wikidata-based benchmarks, with efficiency advantages. The work provides a thorough ablation study across EL, NER, relation detection, and error analysis, demonstrates strong performance on rare entities, and delivers an open-source baseline for KGQA that is robust and scalable. Overall, the paper demonstrates that a carefully designed, component-based KGQA pipeline can rival more resource-intensive neural models for simple questions and offers practical guidance for deploying KGQA systems with real-world KG data.

Abstract

While being one of the most popular question types, simple questions such as "Who is the author of Cinderella?", are still not completely solved. Surprisingly, even the most powerful modern Large Language Models are prone to errors when dealing with such questions, especially when dealing with rare entities. At the same time, as an answer may be one hop away from the question entity, one can try to develop a method that uses structured knowledge graphs (KGs) to answer such questions. In this paper, we introduce Konstruktor - an efficient and robust approach that breaks down the problem into three steps: (i) entity extraction and entity linking, (ii) relation prediction, and (iii) querying the knowledge graph. Our approach integrates language models and knowledge graphs, exploiting the power of the former and the interpretability of the latter. We experiment with two named entity recognition and entity linking methods and several relation detection techniques. We show that for relation detection, the most challenging step of the workflow, a combination of relation classification/generation and ranking outperforms other methods. We report Konstruktor's strong results on four datasets.

Figures (2)

• Figure 1: An overview of the Konstruktor method workflow for simple question answering.
• Figure 2: Accuracy depending on the minimum popularity of the question and answer entities. Popularity is measured as corresponding Wikipedia pageviews.