Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall

Jiaqing Yuan, Lin Pan, Chung-Wei Hang, Jiang Guo, Jiarong Jiang, Bonan Min, Patrick Ng, Zhiguo Wang

TL;DR

This work addresses how well LLMs recall factual knowledge learned during pretraining and how factors such as model size, training regime, and prompting affect recall. It introduces FACT-Bench, a diverse, grounded benchmark built from Wikidata triplets with Wikipedia grounding, spanning 20 domains, 134 properties, and three answer types, and validated via human curation to estimate upper bounds. Through large-scale benchmarking of 31 models across 10 families, the study finds that pretraining lone often outperforms instruction-tuned variants, scaling improves performance, and GPT-4 remains far from an upper bound; it also reveals that counterfactual in-context exemplars significantly undermine recall in large models and that fine-tuning on known knowledge can enhance recall. The results offer practical guidance for prompting and fine-tuning LLMs and establish FACT-Bench as a rigorous, public resource for evaluating factual recall. Overall, the work highlights the importance of grounding prompts and training data in reliable knowledge sources to mitigate hallucinations in LLMs.

Abstract

Large language models (LLMs) have shown remarkable performance on a variety of NLP tasks, and are being rapidly adopted in a wide range of use cases. It is therefore of vital importance to holistically evaluate the factuality of their generated outputs, as hallucinations remain a challenging issue. In this work, we focus on assessing LLMs' ability to recall factual knowledge learned from pretraining, and the factors that affect this ability. To that end, we construct FACT-BENCH, a representative benchmark covering 20 domains, 134 property types, 3 answer types, and different knowledge popularity levels. We benchmark 31 models from 10 model families and provide a holistic assessment of their strengths and weaknesses. We observe that instruction-tuning hurts knowledge recall, as pretraining-only models consistently outperform their instruction-tuned counterparts, and positive effects of model scaling, as larger models outperform smaller ones for all model families. However, the best performance from GPT-4 still represents a large gap with the upper-bound. We additionally study the role of in-context exemplars using counterfactual demonstrations, which lead to significant degradation of factual knowledge recall for large models. By further decoupling model known and unknown knowledge, we find the degradation is attributed to exemplars that contradict a model's known knowledge, as well as the number of such exemplars. Lastly, we fine-tune LLaMA-7B in different settings of known and unknown knowledge. In particular, fine-tuning on a model's known knowledge is beneficial, and consistently outperforms fine-tuning on unknown and mixed knowledge. We will make our benchmark publicly available.

Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall

TL;DR

This work addresses how well LLMs recall factual knowledge learned during pretraining and how factors such as model size, training regime, and prompting affect recall. It introduces FACT-Bench, a diverse, grounded benchmark built from Wikidata triplets with Wikipedia grounding, spanning 20 domains, 134 properties, and three answer types, and validated via human curation to estimate upper bounds. Through large-scale benchmarking of 31 models across 10 families, the study finds that pretraining lone often outperforms instruction-tuned variants, scaling improves performance, and GPT-4 remains far from an upper bound; it also reveals that counterfactual in-context exemplars significantly undermine recall in large models and that fine-tuning on known knowledge can enhance recall. The results offer practical guidance for prompting and fine-tuning LLMs and establish FACT-Bench as a rigorous, public resource for evaluating factual recall. Overall, the work highlights the importance of grounding prompts and training data in reliable knowledge sources to mitigate hallucinations in LLMs.

Abstract

Large language models (LLMs) have shown remarkable performance on a variety of NLP tasks, and are being rapidly adopted in a wide range of use cases. It is therefore of vital importance to holistically evaluate the factuality of their generated outputs, as hallucinations remain a challenging issue. In this work, we focus on assessing LLMs' ability to recall factual knowledge learned from pretraining, and the factors that affect this ability. To that end, we construct FACT-BENCH, a representative benchmark covering 20 domains, 134 property types, 3 answer types, and different knowledge popularity levels. We benchmark 31 models from 10 model families and provide a holistic assessment of their strengths and weaknesses. We observe that instruction-tuning hurts knowledge recall, as pretraining-only models consistently outperform their instruction-tuned counterparts, and positive effects of model scaling, as larger models outperform smaller ones for all model families. However, the best performance from GPT-4 still represents a large gap with the upper-bound. We additionally study the role of in-context exemplars using counterfactual demonstrations, which lead to significant degradation of factual knowledge recall for large models. By further decoupling model known and unknown knowledge, we find the degradation is attributed to exemplars that contradict a model's known knowledge, as well as the number of such exemplars. Lastly, we fine-tune LLaMA-7B in different settings of known and unknown knowledge. In particular, fine-tuning on a model's known knowledge is beneficial, and consistently outperforms fine-tuning on unknown and mixed knowledge. We will make our benchmark publicly available.
Paper Structure (26 sections, 7 figures, 13 tables)

This paper contains 26 sections, 7 figures, 13 tables.

Table of Contents

  1. Introduction
  2. FACT-Bench
  3. Dataset Construction
  4. Dataset Statistics and Evaluation Metrics
  5. Dataset Validation
  6. Benchmarking LLMs
  7. Experimental Setup
  8. Results
  9. Fine-grained Evaluation
  10. The Role of In-context Exemplars
  11. Counterfactual ICL
  12. Counterfactual ICL with known and unknown knowledge
  13. Fine-tuning
  14. Regular fine-tuning
  15. Counterfactual fine-tuning
  16. ...and 11 more sections

Figures (7)

  • Figure 1: 10-shot EM by knowledge popularity. Knowledge popularity is a strong predictor of knowledge recall. LLMs struggle with long-tail entities (Bottom-25%) as shown by the large gap with popular entities (Top-25%).
  • Figure 2: 10-shot EM by property type. LLMs do well on certain property types, such as country-related properties, while struggle on other property types, such as date-related properties. Due to space, we show results for GPT and LLaMA models, and the most common property types from the full set of 134 property types.
  • Figure 3: 10-shot EM by domain. Compared to knowledge popularity and property type, domain is less predictive of knowledge recall as model performances across different domains are more flat. Due to space, we show results for GPT and LLaMA models.
  • Figure 4: 10-shot EM by answer type. LLMs are less capable on date and numerical knowledge. Due to space, we show results for GPT and LLaMA models.
  • Figure 5: LLaMA zero-to-10-shot results by EM.
  • ...and 2 more figures