LLM-assisted Knowledge Graph Engineering: Experiments with ChatGPT
Lars-Peter Meyer, Claus Stadler, Johannes Frey, Norman Radtke, Kurt Junghanns, Roy Meissner, Gordian Dziwis, Kirill Bulert, Michael Martin
TL;DR
The paper investigates how LLMs, particularly ChatGPT, can support Knowledge Graph Engineering tasks such as ontology generation, SPARQL query generation, and KG exploration. It conducts an empirical program using ChatGPT-3.5-turbo and GPT-4 across a small custom KG, the Mondial KG, and fact-sheet extraction to assess capabilities and limitations. Findings show that LLMs can produce syntactically correct SPARQL and constructive visualizations, but often yield incorrect results, fail to align with ontologies, or fail to execute queries, highlighting hallucination risks. The work underscores the need for robust validation, reproducibility, and open training data to advance reliable LLM-assisted KGE.
Abstract
Knowledge Graphs (KG) provide us with a structured, flexible, transparent, cross-system, and collaborative way of organizing our knowledge and data across various domains in society and industrial as well as scientific disciplines. KGs surpass any other form of representation in terms of effectiveness. However, Knowledge Graph Engineering (KGE) requires in-depth experiences of graph structures, web technologies, existing models and vocabularies, rule sets, logic, as well as best practices. It also demands a significant amount of work. Considering the advancements in large language models (LLMs) and their interfaces and applications in recent years, we have conducted comprehensive experiments with ChatGPT to explore its potential in supporting KGE. In this paper, we present a selection of these experiments and their results to demonstrate how ChatGPT can assist us in the development and management of KGs.