From Hallucinations to Facts: Enhancing Language Models with Curated Knowledge Graphs

TL;DR

This work tackles language-model hallucination by grounding generation in a curated knowledge graph (KG) built from Wikipedia, focused on environmental sustainability. The authors detail a full KG creation pipeline (topic-to-subtopic expansion, raw-text extraction, relation and tail-entity selection, and triple generation) and integrate the KG via embedding-based matching, converting triples into natural-language sentences, and augmenting the model output. They evaluate multiple models and three grounding strategies (RAG, IC, CoV) with a comprehensive set of automatic and human metrics, showing consistent reductions in hallucinations and improved factual grounding, particularly for Retrieval-Augmented Generation. The study demonstrates the practical potential of domain-specific, curated KGs to enhance factuality and context relevance in AI-assisted tasks with significant implications for trustworthy, data-grounded dialogue and QA systems.

Abstract

Hallucination, a persistent challenge plaguing language models, undermines their efficacy and trustworthiness in various natural language processing endeavors by generating responses that deviate from factual accuracy or coherence. This paper addresses language model hallucination by integrating curated knowledge graph (KG) triples to anchor responses in empirical data. We meticulously select and integrate relevant KG triples tailored to specific contexts, enhancing factual grounding and alignment with input. Our contribution involves constructing a comprehensive KG repository from Wikipedia and refining data to spotlight essential information for model training. By imbuing language models with access to this curated knowledge, we aim to generate both linguistically fluent responses and deeply rooted in factual accuracy and context relevance. This integration mitigates hallucinations by providing a robust foundation of information, enabling models to draw upon a rich reservoir of factual data during response generation. Experimental evaluations demonstrate the effectiveness of multiple approaches in reducing hallucinatory responses, underscoring the role of curated knowledge graphs in improving the reliability and trustworthiness of language model outputs.

Figures (2)

• Figure 1: Example demonstrating model hallucinations. The response corresponds to the Llama3 model. The actual value is 8 gallons of water, on average.
• Figure 2: Example demonstrating the KG creation process