Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Impact of Negated Text on Hallucination with Large Language Models

Jaehyung Seo, Hyeonseok Moon, Heuiseok Lim

TL;DR

This work investigates how negated text affects hallucination detection in large language models by introducing NegHalu, a dataset built from pre- and post-negated versions of established hallucination benchmarks. Through comprehensive experiments with multiple open-source LLMs, the study reveals systematic degradation and biases in hallucination judgments when handling negation, along with limited improvements from in-context learning, chain-of-thought, and knowledge editing. Lens analyses show that models fail to internalize negation as a logical operation, treating it largely as a lexical modifier. The findings underscore the need for architectural and methodological advances to robustly handle negation and maintain faithful outputs across diverse tasks and domains.

Abstract

Recent studies on hallucination in large language models (LLMs) have been actively progressing in natural language processing. However, the impact of negated text on hallucination with LLMs remains largely unexplored. In this paper, we set three important yet unanswered research questions and aim to address them. To derive the answers, we investigate whether LLMs can recognize contextual shifts caused by negation and still reliably distinguish hallucinations comparable to affirmative cases. We also design the NegHalu dataset by reconstructing existing hallucination detection datasets with negated expressions. Our experiments demonstrate that LLMs struggle to detect hallucinations in negated text effectively, often producing logically inconsistent or unfaithful judgments. Moreover, we trace the internal state of LLMs as they process negated inputs at the token level and reveal the challenges of mitigating their unintended effects.

The Impact of Negated Text on Hallucination with Large Language Models

TL;DR

This work investigates how negated text affects hallucination detection in large language models by introducing NegHalu, a dataset built from pre- and post-negated versions of established hallucination benchmarks. Through comprehensive experiments with multiple open-source LLMs, the study reveals systematic degradation and biases in hallucination judgments when handling negation, along with limited improvements from in-context learning, chain-of-thought, and knowledge editing. Lens analyses show that models fail to internalize negation as a logical operation, treating it largely as a lexical modifier. The findings underscore the need for architectural and methodological advances to robustly handle negation and maintain faithful outputs across diverse tasks and domains.

Abstract

Recent studies on hallucination in large language models (LLMs) have been actively progressing in natural language processing. However, the impact of negated text on hallucination with LLMs remains largely unexplored. In this paper, we set three important yet unanswered research questions and aim to address them. To derive the answers, we investigate whether LLMs can recognize contextual shifts caused by negation and still reliably distinguish hallucinations comparable to affirmative cases. We also design the NegHalu dataset by reconstructing existing hallucination detection datasets with negated expressions. Our experiments demonstrate that LLMs struggle to detect hallucinations in negated text effectively, often producing logically inconsistent or unfaithful judgments. Moreover, we trace the internal state of LLMs as they process negated inputs at the token level and reveal the challenges of mitigating their unintended effects.
Paper Structure (41 sections, 4 figures, 16 tables)

This paper contains 41 sections, 4 figures, 16 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Negated Text in LLMs
  4. Hallucination in LLMs
  5. NegHalu
  6. Source Datasets
  7. Post Negation
  8. Data Verification
  9. Qualitative Analysis
  10. Human Evaluation
  11. Data Statistics and Label Balancing
  12. Experimental Setup
  13. Models
  14. Datasets and Evaluation
  15. Lens Observation
  16. ...and 26 more sections

Figures (4)

  • Figure 1: Examples illustrating how negation can flip the hallucination label. The negated response introduces or resolves contradictions with the given knowledge.
  • Figure 2: Logit lens results showing probability shifts for pre- and post-negated examples in the HaluEval subsets of NegHalu. Each curve tracks the probability of the first output token across the layers of the model, comparing pre-negated and post-negated inputs. The blue curves represent scenarios with pre-negated inputs, while the red curves indicate scenarios with post-negated inputs.
  • Figure 3: Performance comparison of Llama3 across NegHalu subsets under negated knowledge updates using AlphaEdit and two different target corpus.
  • Figure 4: Logit lens results showing probability shifts for pre- and post-negated examples in the HaluEval subsets of NegHalu. The blue curves represent cases where the model generates the correct answer, while the red curves indicate cases where the model generates an incorrect answer.