On A Scale From 1 to 5: Quantifying Hallucination in Faithfulness Evaluation

TL;DR

This work tackles measuring hallucination in guided NLG by proposing automated faithfulness evaluation through two complementary approaches: rubric-based LLM scoring and NLI-based entailment evaluation, augmented with synthetic hallucination data. The authors compare multiple LLMs and NLI models across four travel-domain datasets, showing that GPT-4 offers the strongest zero-shot faithfulness judgments while synthetic data can substantially boost NLI performance; they additionally introduce a percentile hallucination metric to visualize sensitivity. A key practical outcome is guidance on deploying faithful NLG systems, highlighting the trade-offs between accuracy, latency, and cost, and proposing a two-stage evaluation workflow. The study also discusses limitations and directions for rubric refinement and broader domain generalization.

Abstract

Hallucination has been a popular topic in natural language generation (NLG). In real-world applications, unfaithful content can result in poor data quality or loss of trust from end users. Thus, it is crucial to fact-check before adopting NLG for production usage, which can be expensive if done manually. In this paper, we investigate automated faithfulness evaluation in guided NLG. We developed a rubric template and used large language models (LLMs) to score the generation on quantifiable scales. We compared popular LLMs as well as widely adopted natural language inference (NLI) models in scoring quality and sensitivity. In addition, we developed methods for the generation of synthetic unfaithful data, as well as heuristics to quantify the percentage of hallucination. Our results on 4 travel-domain industry dataset show that GPT-4 can provide accurate judgement and explanation of whether a source and a generation are factually consistent. Furthermore, we found that tuning NLI models on synthetic data can improve performance. Lastly, we present insights on the latency and cost of deploying such a system.

Figures (3)

• Figure 1: Guided NLG showcasing a text generation example. Given a prompting template and grounding data, hallucination can happen in intrinsic or extrinsic ways. Both can be harmful to a product because the generated contents can either be wrong or unverified.
• Figure 2: Residuals computed as the absolute error between expected and predicted scores. The scores range from "1-highly unfaithful" to "5-highly faithful". The y-axis denotes the absolute value of the errors with a shorter bar indicating better model performance at a corresponding task. Tasks are arranged in the order of (a) Gold full length; (b) Gold sentence-level; (c) Intrinsic sentence-level; (d) Extrinsic sentence-level; and (e) All tasks. The expected score for gold data (a) and (b) should be 5 whereas for hallucination data (c) and (d) should be 1. Overall, GPT-4 is the most capable of scoring both gold and hallucination data. LLMs with less than 10B parameters seem to struggle with different aspects of the scoring. Tuning on HHEM with additional synthetic examples also improved overall performance.
• Figure 3: Score progression on different hallucination percentage. (top) Intrinsic hallucination. (bottom) Extrinsic hallucination. The scores range from "1-highly unfaithful" to "5-highly faithful". The subset contains 51 samples with only 5-sentence hypothesis. Starting with the gold hypothesis (0%), the hallucination percentage is increased by 20% at every step, until all sentences are hallucinating sentences (100%). The "SO" suffix indicates "score-only" results, which the prompt was adjusted to exclude the "reasoning" in the output. When the hypothesis contains 0% hallucination, the expected score should be "5-highly faithful"; and when the hypothesis contains 100% hallucination, the expected score should be "1-highly unfaithful".