Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Don't Believe Everything You Read: Enhancing Summarization Interpretability through Automatic Identification of Hallucinations in Large Language Models

Priyesh Vakharia, Devavrat Joshi, Meenal Chavan, Dhananjay Sonawane, Bhrigu Garg, Parsa Mazaheri

TL;DR

The paper tackles hallucinations in LLM-based dialogue summarization by introducing token-level hallucination tagging to improve interpretability and faithfulness. It builds on the SAMSum dataset with CONFIT-derived token-level annotations (and MetaSAMSum) to train proxy models and to explore both separate and joint modeling approaches for summarization and faithfulness tagging. Three main contributions emerge: (i) a token-level faithfulness taxonomy and enhanced dataset, (ii) a proxy model and prompting strategies (including GPT-4/LLama1-family experiments) to identify and explain hallucinations, and (iii) a joint model that generates summaries while predicting token-level faithfulness tags, showing measurable ROUGE gains and improved factuality in some cases. The findings advance interpretability and reliability in automatic summarization and highlight the need for better evaluation metrics beyond ROUGE to assess faithfulness and factuality in generated content.

Abstract

Large Language Models (LLMs) are adept at text manipulation -- tasks such as machine translation and text summarization. However, these models can also be prone to hallucination, which can be detrimental to the faithfulness of any answers that the model provides. Recent works in combating hallucinations in LLMs deal with identifying hallucinated sentences and categorizing the different ways in which models hallucinate. This paper takes a deep dive into LLM behavior with respect to hallucinations, defines a token-level approach to identifying different kinds of hallucinations, and further utilizes this token-level tagging to improve the interpretability and faithfulness of LLMs in dialogue summarization tasks. Through this, the paper presents a new, enhanced dataset and a new training paradigm.

Don't Believe Everything You Read: Enhancing Summarization Interpretability through Automatic Identification of Hallucinations in Large Language Models

TL;DR

The paper tackles hallucinations in LLM-based dialogue summarization by introducing token-level hallucination tagging to improve interpretability and faithfulness. It builds on the SAMSum dataset with CONFIT-derived token-level annotations (and MetaSAMSum) to train proxy models and to explore both separate and joint modeling approaches for summarization and faithfulness tagging. Three main contributions emerge: (i) a token-level faithfulness taxonomy and enhanced dataset, (ii) a proxy model and prompting strategies (including GPT-4/LLama1-family experiments) to identify and explain hallucinations, and (iii) a joint model that generates summaries while predicting token-level faithfulness tags, showing measurable ROUGE gains and improved factuality in some cases. The findings advance interpretability and reliability in automatic summarization and highlight the need for better evaluation metrics beyond ROUGE to assess faithfulness and factuality in generated content.

Abstract

Large Language Models (LLMs) are adept at text manipulation -- tasks such as machine translation and text summarization. However, these models can also be prone to hallucination, which can be detrimental to the faithfulness of any answers that the model provides. Recent works in combating hallucinations in LLMs deal with identifying hallucinated sentences and categorizing the different ways in which models hallucinate. This paper takes a deep dive into LLM behavior with respect to hallucinations, defines a token-level approach to identifying different kinds of hallucinations, and further utilizes this token-level tagging to improve the interpretability and faithfulness of LLMs in dialogue summarization tasks. Through this, the paper presents a new, enhanced dataset and a new training paradigm.
Paper Structure (36 sections, 4 equations, 13 figures, 5 tables)

This paper contains 36 sections, 4 equations, 13 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Data
  3. Data annotation and guidelines
  4. Data Analysis
  5. Proxy Model for Hallucination Tagging
  6. Baseline
  7. Novelty
  8. Training Setup
  9. Experiment Plan
  10. Fine-tuning
  11. GPT4 Few Shot Tagging
  12. Evaluation
  13. Results
  14. Experiments with Large Language Models
  15. Goal
  16. ...and 21 more sections

Figures (13)

  • Figure 1: Example of dialogue from the SAMSum corpus.
  • Figure 2: Distribution of total turns in the train data
  • Figure 3: Length of summaries in the train data
  • Figure 4: Frequency of Most Common Errors
  • Figure 5: Frequency of Most Common Errors per Model
  • ...and 8 more figures