Knowledge Graphs for Enhancing Large Language Models in Entity Disambiguation

TL;DR

The paper tackles zero-shot entity disambiguation (ED) in NLP by addressing LLM hallucination and knowledge gaps through Knowledge Graphs (KGs). It introduces a two-step method that builds a DAG $G$ from a KG rooted at $Thing$, connects candidate leaves to their classes, and uses an $LCA$-driven pruning process to narrow to a single entity, optionally retrieving entity descriptions for disambiguation. Across ten ED datasets, the KG-enhanced prompting approach outperforms non-enhanced and description-only baselines and shows greater adaptability than task-specific models, with performance improving as KG expressivity increases (e.g., YAGO vs DBpedia). An extensive error analysis further informs limitations and outlines directions for improving robustness with more capable LLMs and richer KGs.

Abstract

Recent advances in Large Language Models (LLMs) have positioned them as a prominent solution for Natural Language Processing tasks. Notably, they can approach these problems in a zero or few-shot manner, thereby eliminating the need for training or fine-tuning task-specific models. However, LLMs face some challenges, including hallucination and the presence of outdated knowledge or missing information from specific domains in the training data. These problems cannot be easily solved by retraining the models with new data as it is a time-consuming and expensive process. To mitigate these issues, Knowledge Graphs (KGs) have been proposed as a structured external source of information to enrich LLMs. With this idea, in this work we use KGs to enhance LLMs for zero-shot Entity Disambiguation (ED). For that purpose, we leverage the hierarchical representation of the entities' classes in a KG to gradually prune the candidate space as well as the entities' descriptions to enrich the input prompt with additional factual knowledge. Our evaluation on popular ED datasets shows that the proposed method outperforms non-enhanced and description-only enhanced LLMs, and has a higher degree of adaptability than task-specific models. Furthermore, we conduct an error analysis and discuss the impact of the leveraged KG's semantic expressivity on the ED performance.

Figures (4)

• Figure 1: Overview of the two steps of our approach.
• Figure 2: Overview of the steps for the creation of the DAG.
• Figure 3: Example of the three different configurations of the LCA's direct successors.
• Figure 4: Class representation of the entity Barcelona in DBpedia (left) and YAGO (right) KGs.