The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models

TL;DR

This work provides a comprehensive empirical study of factuality hallucinations in LLMs across detection, source, and mitigation. It introduces HaluEval 2.0, a large, multi-domain benchmark, and a two-step detection framework based on fact extraction and judgement. Through systematic analysis of pre-training, supervision fine-tuning, prompting, and inference, it reveals nuanced, domain- and model-dependent effects and evaluates multiple mitigation strategies (RLHF, retrieval augmentation, self-reflexion, decoding, and prompt engineering). The findings offer practical guidance for deploying LLMs more reliably and underscore the need for domain-aware strategies and robust evaluation benchmarks.

Abstract

In the era of large language models (LLMs), hallucination (i.e., the tendency to generate factually incorrect content) poses great challenge to trustworthy and reliable deployment of LLMs in real-world applications. To tackle the LLM hallucination, three key questions should be well studied: how to detect hallucinations (detection), why do LLMs hallucinate (source), and what can be done to mitigate them (mitigation). To address these challenges, this work presents a systematic empirical study on LLM hallucination, focused on the the three aspects of hallucination detection, source and mitigation. Specially, we construct a new hallucination benchmark HaluEval 2.0, and designs a simple yet effective detection method for LLM hallucination. Furthermore, we zoom into the different training or utilization stages of LLMs and extensively analyze the potential factors that lead to the LLM hallucination. Finally, we implement and examine a series of widely used techniques to mitigate the hallucinations in LLMs. Our work has led to several important findings to understand the hallucination origin and mitigate the hallucinations in LLMs. Our code and data can be accessed at https://github.com/RUCAIBox/HaluEval-2.0.

Figures (8)

• Figure 1: Baichuan 2 (7B) task accuracy (%) in three benchmarks and average hallucination rate (%) in five domains with respect to billions of pretraining tokens. We average the macro and micro hallucination rates. The accuracy results in (a) are copied from Baichuan 2 baichuan2.
• Figure 2: Ratios of pre-training data sources (figure copied from the LLM survey article zhao2023survey).
• Figure 3: Evaluation results of ChatGPT and Llama 2-Chat (7B). The red line denotes the frequency of entities, and the blue and green bar denotes the average hallucination rate (%) for each group of entities.
• Figure 4: The average hallucination rate (%) of those responses and questions by ChatGPT for each score of the three properties, i.e., readability, formality, and concreteness, in five domains. Some values are zero because there are no scores from humans. The 5-point denotes "very satisfying" and the 1-point denotes "very terrible".
• Figure 5: Average hallucination rate (%) with varying $p$ in top-$p$ sampling.