Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Strong hallucinations from negation and how to fix them

Nicholas Asher, Swarnadeep Bhar

TL;DR

The paper defines strong hallucinations as reasoning errors arising from how language models represent negation, showing that standard LM objectives fail to respect logical semantics. It introduces a hybrid approach, Lambda, which treats negation as an operator over latent representations that constrains how continuation distributions evolve, mapping $\mu(B|A)$ to $1-\mu(B|A)$ and enforcing coherent conjunction and implication behavior. Empirically, Lambda yields substantial improvements on negation-sensitive tasks, including masked knowledge retrieval and natural language inference, without requiring negative data for training. This semantically grounded, post hoc constraint offers a scalable path to reducing logical errors in LMs and can be extended to other logical operators and interactive learning setups like RLHF.

Abstract

Despite great performance on many tasks, language models (LMs) still struggle with reasoning, sometimes providing responses that cannot possibly be true because they stem from logical incoherence. We call such responses \textit{strong hallucinations} and prove that they follow from an LM's computation of its internal representations for logical operators and outputs from those representations. Focusing on negation, we provide a novel solution in which negation is treated not as another element of a latent representation, but as \textit{an operation over an LM's latent representations that constrains how they may evolve}. We show that our approach improves model performance in cloze prompting and natural language inference tasks with negation without requiring training on sparse negative data.

Strong hallucinations from negation and how to fix them

TL;DR

The paper defines strong hallucinations as reasoning errors arising from how language models represent negation, showing that standard LM objectives fail to respect logical semantics. It introduces a hybrid approach, Lambda, which treats negation as an operator over latent representations that constrains how continuation distributions evolve, mapping to and enforcing coherent conjunction and implication behavior. Empirically, Lambda yields substantial improvements on negation-sensitive tasks, including masked knowledge retrieval and natural language inference, without requiring negative data for training. This semantically grounded, post hoc constraint offers a scalable path to reducing logical errors in LMs and can be extended to other logical operators and interactive learning setups like RLHF.

Abstract

Despite great performance on many tasks, language models (LMs) still struggle with reasoning, sometimes providing responses that cannot possibly be true because they stem from logical incoherence. We call such responses \textit{strong hallucinations} and prove that they follow from an LM's computation of its internal representations for logical operators and outputs from those representations. Focusing on negation, we provide a novel solution in which negation is treated not as another element of a latent representation, but as \textit{an operation over an LM's latent representations that constrains how they may evolve}. We show that our approach improves model performance in cloze prompting and natural language inference tasks with negation without requiring training on sparse negative data.
Paper Structure (12 sections, 5 theorems, 7 equations, 1 figure, 3 tables, 4 algorithms)

This paper contains 12 sections, 5 theorems, 7 equations, 1 figure, 3 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. LMs, negation and strong hallucinations
  4. A hybrid treatment of logical operators for LMs
  5. Negation as a constraint on continuations
  6. Experiments with our approach on Q&A, cloze prompts, NLI and negation
  7. Q&A tasks.
  8. Masked Knowledge Retrieval.
  9. NLI.
  10. Related work
  11. Conclusions
  12. Limitations

Key Result

Proposition 1

Every LM $\hat{f}$ whose outputs are governed by $\mu_{\hat{f}}$ and Definition affirm must strongly hallucinate if either: (i) $\sum_{x \in V^n \setminus \{\phi, \neg \phi\}} \mu_{\hat{f}}(x) > 0$; (ii) $\mu_{\hat{f}}$ assigns values to strings in $V^n$ that are logical truths or deductively valid

Figures (1)

  • Figure 1: Histogram plot of cossim values for positive and negative contexts with BERT-large (above) BERT-base (below) before (red) and after fine-tuning (blue).

Theorems & Definitions (11)

  • Definition 1
  • Definition 2
  • Proposition 1
  • Proposition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Proposition 3
  • Proposition 4
  • ...and 1 more