Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GraphEval: A Knowledge-Graph Based LLM Hallucination Evaluation Framework

Hannah Sansford, Nicholas Richardson, Hermina Petric Maretic, Juba Nait Saada

TL;DR

GraphEval introduces a knowledge-graph based framework to detect and localize hallucinations in LLM outputs conditioned on grounding context. It constructs a KG from the LLM output and uses an NLI model on each triple to assess grounding consistency, reporting inconsistent triples for explainability. When combined with state-of-the-art NLI models, GraphEval improves balanced accuracy on hallucination benchmarks by about 6.2 points on average and enables GraphCorrect to correct many hallucinations while preserving text similarity. This approach offers a scalable, explainable alternative to fully LLM-based evaluation and suggests a promising route for targeted correction of hallucinations.

Abstract

Methods to evaluate Large Language Model (LLM) responses and detect inconsistencies, also known as hallucinations, with respect to the provided knowledge, are becoming increasingly important for LLM applications. Current metrics fall short in their ability to provide explainable decisions, systematically check all pieces of information in the response, and are often too computationally expensive to be used in practice. We present GraphEval: a hallucination evaluation framework based on representing information in Knowledge Graph (KG) structures. Our method identifies the specific triples in the KG that are prone to hallucinations and hence provides more insight into where in the response a hallucination has occurred, if at all, than previous methods. Furthermore, using our approach in conjunction with state-of-the-art natural language inference (NLI) models leads to an improvement in balanced accuracy on various hallucination benchmarks, compared to using the raw NLI models. Lastly, we explore the use of GraphEval for hallucination correction by leveraging the structure of the KG, a method we name GraphCorrect, and demonstrate that the majority of hallucinations can indeed be rectified.

GraphEval: A Knowledge-Graph Based LLM Hallucination Evaluation Framework

TL;DR

GraphEval introduces a knowledge-graph based framework to detect and localize hallucinations in LLM outputs conditioned on grounding context. It constructs a KG from the LLM output and uses an NLI model on each triple to assess grounding consistency, reporting inconsistent triples for explainability. When combined with state-of-the-art NLI models, GraphEval improves balanced accuracy on hallucination benchmarks by about 6.2 points on average and enables GraphCorrect to correct many hallucinations while preserving text similarity. This approach offers a scalable, explainable alternative to fully LLM-based evaluation and suggests a promising route for targeted correction of hallucinations.

Abstract

Methods to evaluate Large Language Model (LLM) responses and detect inconsistencies, also known as hallucinations, with respect to the provided knowledge, are becoming increasingly important for LLM applications. Current metrics fall short in their ability to provide explainable decisions, systematically check all pieces of information in the response, and are often too computationally expensive to be used in practice. We present GraphEval: a hallucination evaluation framework based on representing information in Knowledge Graph (KG) structures. Our method identifies the specific triples in the KG that are prone to hallucinations and hence provides more insight into where in the response a hallucination has occurred, if at all, than previous methods. Furthermore, using our approach in conjunction with state-of-the-art natural language inference (NLI) models leads to an improvement in balanced accuracy on various hallucination benchmarks, compared to using the raw NLI models. Lastly, we explore the use of GraphEval for hallucination correction by leveraging the structure of the KG, a method we name GraphCorrect, and demonstrate that the majority of hallucinations can indeed be rectified.
Paper Structure (24 sections, 1 figure, 4 tables)

This paper contains 24 sections, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Problem statement
  3. Related work
  4. GraphEval: Our evaluation method
  5. Construction of KGs using LLMs
  6. GraphCorrect: Correction of hallucinations with GraphEval
  7. Experiments
  8. Benchmarks
  9. SummEval
  10. QAGS
  11. NLI models in GraphEval
  12. HHEM
  13. TRUE
  14. TrueTeacher
  15. Experimental settings
  16. ...and 9 more sections

Figures (1)

  • Figure 1: A visualisation of the GraphEval approach. First, the LLM output is fed into the KG construction prompt to produce the KG depicted on the right. Next, each individual triple in the KG is fed into an out-of-the-box hallucination detection method, such as an NLI model, and compared to the provided context for inconsistencies. Finally, any triples that are flagged as inconsistent are returned to the user, along with the overall hallucination decision.