DHI: Leveraging Diverse Hallucination Induction for Enhanced Contrastive Factuality Control in Large Language Models

TL;DR

DHI tackles the persistent problem of hallucinations in large language models by introducing a diverse hallucination induction framework that trains an Evil LLM to generate varied but plausible errors through a modified loss and a causal masking strategy. It couples this Evil model with a Positive model in a selective, adaptive contrastive decoding scheme that only applies when the Positive model is uncertain, improving factuality without harming correct predictions. Key contributions include a novel loss formulation $L_{\text{evil}}$ and a causal attention masking adaptation, plus an adaptive rationality constraint during inference, all validated by substantial gains on TruthfulQA and FACTSCORE benchmarks. The results demonstrate that diversity in induced hallucinations and targeted contrastive decoding yield robust factuality improvements, with ablations confirming the importance of each component for handling complex metrics like MC3.

Abstract

Large language models (LLMs) frequently produce inaccurate or fabricated information, known as "hallucinations," which compromises their reliability. Existing approaches often train an "Evil LLM" to deliberately generate hallucinations on curated datasets, using these induced hallucinations to guide contrastive decoding against a reliable "positive model" for hallucination mitigation. However, this strategy is limited by the narrow diversity of hallucinations induced, as Evil LLMs trained on specific error types tend to reproduce only these particular patterns, thereby restricting their overall effectiveness. To address these limitations, we propose DHI (Diverse Hallucination Induction), a novel training framework that enables the Evil LLM to generate a broader range of hallucination types without relying on pre-annotated hallucination data. DHI employs a modified loss function that down-weights the generation of specific factually correct tokens, encouraging the Evil LLM to produce diverse hallucinations at targeted positions while maintaining overall factual content. Additionally, we introduce a causal attention masking adaptation to reduce the impact of this penalization on the generation of subsequent tokens. During inference, we apply an adaptive rationality constraint that restricts contrastive decoding to tokens where the positive model exhibits high confidence, thereby avoiding unnecessary penalties on factually correct tokens. Extensive empirical results show that DHI achieves significant performance gains over other contrastive decoding-based approaches across multiple hallucination benchmarks.

Figures (2)

• Figure 1: Contrastive decoding can effectively alleviate hallucinations. ICD trains its Evil LLM by rewarding specific, predefined hallucinated tokens, which may bias the model toward producing a limited range of error patterns and affect the generation of subsequent tokens. In contrast, our proposed DHI improves upon ICD by downweighting factually correct tokens, encouraging the generation of more diverse hallucinations. Additionally, we employ a causal masking strategy to minimize the impact of this penalization on subsequent tokens, thereby preserving coherence in the generated text.
• Figure 2: Improved Causal Attention Mask to mitigate impacts on subsequent tokens during Evil Model training.