Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

What if...?: Thinking Counterfactual Keywords Helps to Mitigate Hallucination in Large Multi-modal Models

Junho Kim, Yeon Ju Kim, Yong Man Ro

TL;DR

This work tackles hallucination in large multi-modal models by injecting counterfactual thinking via self-generated keywords (Counterfactual Inception). It achieves this without parameter updates, using a Plausibility Verification Process (PVP) to filter keywords and condition responses on plausible yet alternative contexts. Across both discriminative and generative benchmarks on open-source and proprietary LMMs, the approach reduces hallucinations and improves grounding to true visual clues while enabling broader contextual understanding. The method thus offers a scalable, training-free mitigation for cross-modal reasoning in real-world deployments.

Abstract

This paper presents a way of enhancing the reliability of Large Multi-modal Models (LMMs) in addressing hallucination, where the models generate cross-modal inconsistent responses. Without additional training, we propose Counterfactual Inception, a novel method that implants counterfactual thinking into LMMs using self-generated counterfactual keywords. Our method is grounded in the concept of counterfactual thinking, a cognitive process where human considers alternative realities, enabling more extensive context exploration. Bridging the human cognition mechanism into LMMs, we aim for the models to engage with and generate responses that span a wider contextual scene understanding, mitigating hallucinatory outputs. We further introduce Plausibility Verification Process (PVP), a simple yet robust keyword constraint that effectively filters out sub-optimal keywords to enable the consistent triggering of counterfactual thinking in the model responses. Comprehensive analyses across various LMMs, including both open-source and proprietary models, corroborate that counterfactual thinking significantly reduces hallucination and helps to broaden contextual understanding based on true visual clues.

What if...?: Thinking Counterfactual Keywords Helps to Mitigate Hallucination in Large Multi-modal Models

TL;DR

This work tackles hallucination in large multi-modal models by injecting counterfactual thinking via self-generated keywords (Counterfactual Inception). It achieves this without parameter updates, using a Plausibility Verification Process (PVP) to filter keywords and condition responses on plausible yet alternative contexts. Across both discriminative and generative benchmarks on open-source and proprietary LMMs, the approach reduces hallucinations and improves grounding to true visual clues while enabling broader contextual understanding. The method thus offers a scalable, training-free mitigation for cross-modal reasoning in real-world deployments.

Abstract

This paper presents a way of enhancing the reliability of Large Multi-modal Models (LMMs) in addressing hallucination, where the models generate cross-modal inconsistent responses. Without additional training, we propose Counterfactual Inception, a novel method that implants counterfactual thinking into LMMs using self-generated counterfactual keywords. Our method is grounded in the concept of counterfactual thinking, a cognitive process where human considers alternative realities, enabling more extensive context exploration. Bridging the human cognition mechanism into LMMs, we aim for the models to engage with and generate responses that span a wider contextual scene understanding, mitigating hallucinatory outputs. We further introduce Plausibility Verification Process (PVP), a simple yet robust keyword constraint that effectively filters out sub-optimal keywords to enable the consistent triggering of counterfactual thinking in the model responses. Comprehensive analyses across various LMMs, including both open-source and proprietary models, corroborate that counterfactual thinking significantly reduces hallucination and helps to broaden contextual understanding based on true visual clues.
Paper Structure (32 sections, 2 equations, 12 figures, 7 tables, 1 algorithm)

This paper contains 32 sections, 2 equations, 12 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Counterfactual Keyword Generation
  5. Counterfactual Inception
  6. $\mathcal{K}_{\text{head}}$: Plausibility Verification Process
  7. Experiments
  8. Experimental Setup
  9. Baselines & Implementation.
  10. Benchmarks and Evaluation Metrics.
  11. Counterfactual Keyword Statistics
  12. Experimental Results
  13. Discriminative Benchmarks.
  14. Generative Benchmarks.
  15. Analysis on Counterfactual Inception
  16. ...and 17 more sections

Figures (12)

  • Figure 1: Counterfactual Inception: LMMs generate counterfactual keywords at the object, attribute, and relation levels, then integrate them with a counterfactual prompt to implant counterfactual thinking to the models. To filter out keywords that are either too similar or too deviated from the visual content, we adopt a robust constraint called PVP.
  • Figure 2: Frequency distribution for the counterfactual keywords. The dashed lines indicate truncation level. We have empirically observed that the keywords in the upper half of the distribution are closer to factual information rather than counterfactual, thus the lower half, excluding extreme low, is set as the criteria. See Fig. \ref{['fig:6']} for the keyword analysis.
  • Figure 3: The statistical results for the number of counterfactual keywords for $6$ baselines and $4$ benchmarks in each three category. Note that the brighter colors in each bar indicates raw keyword count, and the solid colors are the count after adjusting PVP constraint.
  • Figure 4: The cumulative frequency distribution along the scores for COCO dataset with 6 baselines. The dashed lines indicates PVP constraint area.
  • Figure 5: The graphical results of Top-$5$ words occurrence using morphological analysis (NLTK) in counterfactual keywords. Each legend box indicates total number words in object, attribute, and relation keyword category, respectively.
  • ...and 7 more figures