Wikontic: Constructing Wikidata-Aligned, Ontology-Aware Knowledge Graphs with Large Language Models

TL;DR

Wikontic addresses the challenge of converting unstructured text into high-quality, verifiable knowledge graphs by tightly integrating open information extraction with Wikidata ontology constraints. The pipeline extracts triplets with qualifiers, refines them to satisfy schema and domain–range constraints, and deduplicates entities to produce compact, ontology-aligned KGs. When used as the sole knowledge source for multi-hop QA, Wikontic achieves competitive or superior results to retrieval-based baselines and sets state-of-the-art information retention on MINE-1, while remaining token-efficient. These findings demonstrate the feasibility and value of ontology-guided KG construction for scalable, interpretable reasoning with LLMs.

Abstract

Knowledge graphs (KGs) provide structured, verifiable grounding for large language models (LLMs), but current LLM-based systems commonly use KGs as auxiliary structures for text retrieval, leaving their intrinsic quality underexplored. In this work, we propose Wikontic, a multi-stage pipeline that constructs KGs from open-domain text by extracting candidate triplets with qualifiers, enforcing Wikidata-based type and relation constraints, and normalizing entities to reduce duplication. The resulting KGs are compact, ontology-consistent, and well-connected; on MuSiQue, the correct answer entity appears in 96% of generated triplets. On HotpotQA, our triplets-only setup achieves 76.0 F1, and on MuSiQue 59.8 F1, matching or surpassing several retrieval-augmented generation baselines that still require textual context. In addition, Wikontic attains state-of-the-art information-retention performance on the MINE-1 benchmark (86%), outperforming prior KG construction methods. Wikontic is also efficient at build time: KG construction uses less than 1,000 output tokens, about 3$\times$ fewer than AriGraph and $<$1/20 of GraphRAG. The proposed pipeline enhances the quality of the generated KG and offers a scalable solution for leveraging structured knowledge in LLMs.

Figures (6)

• Figure 1: Overview of Wikontic: an ontology-guided pipeline that constructs a Wikidata-aligned KG from text. (1) An LLM extracts candidate (subject, relation, object) triplets (gray). (2) The extracted triplets are then refined using Wikidata’s ontology: entity types are assigned (colored nodes), and relations that violate ontology constraints are corrected or removed. (3) Finally, entities names are normalized and duplicate surface forms are merged. The resulting graph is de-duplicated, ontology-consistent, and ready for downstream tasks.
• Figure 2: Distribution of MINE-1 scores across 100 articles for GraphRAG, KGGen, and Wikontic. Dotted vertical lines are averaged scores. Wikontic scored 84% on average, substantially outperforming GraphRAG 47.80% and KGGen 66%.
• Figure 3: Wikontic produces the most dense KGs for MuSiQue questions. For each question, subgraphs are constructed around its entities, and their sizes are reported relative to the full KG. The figure shows the relative sizes of 1– to 10-hop neighborhoods and the entire connected component containing the question, defined as all nodes reachable from any question node.
• Figure 4: Overview of the multi-stage pipeline for KG extraction from unstructured text. The process consists of (1) LLM-based triplet extraction, (2) ontology-based validation of triplet structure, and (3) entity linking and normalization.
• Figure 5: Ontology-based triplet refinement process. For each extracted triplet, we retrieve and extend candidate entity types using Wikidata’s type hierarchy, identify valid relations allowed to use between extracted entities based on ontology constraints, and re-rank relation candidates using semantic similarity. The final triplet configuration is selected by an LLM.