Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs

Xiaoze Liu, Feijie Wu, Tianyang Xu, Zhuo Chen, Yichi Zhang, Xiaoqian Wang, Jing Gao

TL;DR

GraphEval presents a scalable, knowledge-graph–driven framework for evaluating the factuality of large language models by replacing full-text generation with a lightweight judge that operates on LLM hidden states. By leveraging a large KG (DBpedia) and declarative templates to generate millions of prompts, GraphEval achieves broad coverage across subjects and relations while reducing labeling and compute costs. The approach includes a theoretical generalization bound for the judge, an efficient prompt-encoder pipeline, and extensive experiments showing the judge's predictions align with actual factual correctness, with nuanced insights across LLaMA-2 and Gemma families and relation types. This work offers a practical, domain-agnostic benchmark for factuality that can inform future reliability improvements in LLM outputs and supports scalable, regular evaluation at web-scale KG breadth.

Abstract

The advent of Large Language Models (LLMs) has significantly transformed the AI landscape, enhancing machine learning and AI capabilities. Factuality issue is a critical concern for LLMs, as they may generate factually incorrect responses. In this paper, we propose GraphEval to evaluate an LLM's performance using a substantially large test dataset. Specifically, the test dataset is retrieved from a large knowledge graph with more than 10 million facts without expensive human efforts. Unlike conventional methods that evaluate LLMs based on generated responses, GraphEval streamlines the evaluation process by creating a judge model to estimate the correctness of the answers given by the LLM. Our experiments demonstrate that the judge model's factuality assessment aligns closely with the correctness of the LLM's generated outputs, while also substantially reducing evaluation costs. Besides, our findings offer valuable insights into LLM performance across different metrics and highlight the potential for future improvements in ensuring the factual integrity of LLM outputs. The code is publicly available at https://github.com/xz-liu/GraphEval.

Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs

TL;DR

GraphEval presents a scalable, knowledge-graph–driven framework for evaluating the factuality of large language models by replacing full-text generation with a lightweight judge that operates on LLM hidden states. By leveraging a large KG (DBpedia) and declarative templates to generate millions of prompts, GraphEval achieves broad coverage across subjects and relations while reducing labeling and compute costs. The approach includes a theoretical generalization bound for the judge, an efficient prompt-encoder pipeline, and extensive experiments showing the judge's predictions align with actual factual correctness, with nuanced insights across LLaMA-2 and Gemma families and relation types. This work offers a practical, domain-agnostic benchmark for factuality that can inform future reliability improvements in LLM outputs and supports scalable, regular evaluation at web-scale KG breadth.

Abstract

The advent of Large Language Models (LLMs) has significantly transformed the AI landscape, enhancing machine learning and AI capabilities. Factuality issue is a critical concern for LLMs, as they may generate factually incorrect responses. In this paper, we propose GraphEval to evaluate an LLM's performance using a substantially large test dataset. Specifically, the test dataset is retrieved from a large knowledge graph with more than 10 million facts without expensive human efforts. Unlike conventional methods that evaluate LLMs based on generated responses, GraphEval streamlines the evaluation process by creating a judge model to estimate the correctness of the answers given by the LLM. Our experiments demonstrate that the judge model's factuality assessment aligns closely with the correctness of the LLM's generated outputs, while also substantially reducing evaluation costs. Besides, our findings offer valuable insights into LLM performance across different metrics and highlight the potential for future improvements in ensuring the factual integrity of LLM outputs. The code is publicly available at https://github.com/xz-liu/GraphEval.
Paper Structure (37 sections, 1 theorem, 4 equations, 9 figures, 6 tables)

This paper contains 37 sections, 1 theorem, 4 equations, 9 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Factuality Issue of LLMs
  4. Factuality Evaluation of LLMs
  5. Using KGs in LLMs
  6. Method
  7. Question and Label Collection
  8. Question Generation
  9. Negative sampling
  10. Judge Model
  11. Efficiency
  12. Analysis of Judge model
  13. Evaluation
  14. Experiments
  15. Experiment Setup
  16. ...and 22 more sections

Key Result

Theorem 3.1

Let $\mathcal{H} = \{\mathcal{H}_t, \mathcal{H}_f, \mathcal{H}_{idk}\}$ be a set of hypothesis spaces of VC dimension $d$. If $\mathcal{U}_S, \mathcal{U}_T$ are the samples of size $m$ each, drawn from $\mathcal{D}_S$ and $\mathcal{D}_T$, respectively, then for any $\delta \in (0, 1)$, with probabil where $\lambda = \inf_{h \in \mathcal{H}} \left(L_{\mathcal{D}_S}\left(h\right) + L_{\mathcal{D}_T}

Figures (9)

  • Figure 1: Existing works compared to the proposed GraphEval on factuality evaluation.
  • Figure 2: Overview of the GraphEval framework. Step #1 retrieves KG statements and collect LLM judgments on them. Step #2 trains the judge model which classifies LLM hidden states into three categories. Step #3 evaluates the LLM on all KG statements with the judge model.
  • Figure 3: The LLM's averaged metrics with respect to head entity types and tail entity types
  • Figure 4: Evaluation scores on the judge model's performance on the labeled validation set. P, R, and F are Precision, Recall, and F1 Score.
  • Figure 5: The LLM's correctness with respect to head entity types and tail entity types
  • ...and 4 more figures

Theorems & Definitions (2)

  • Example 3.1
  • Theorem 3.1