Countering the Over-Reliance Trap: Mitigating Object Hallucination for LVLMs via a Self-Validation Framework

TL;DR

The paper analyzes object hallucination in LVLMs as primarily caused by over-reliance on language priors, especially as caption length grows. It introduces Language-Prior-Free Verification (LPFV) to estimate object existence without prior text, and a training-free Self-Validation Framework that uses LPFV to verify objects in multiple candidate captions. Two caption-production strategies, Best-of-N and Filter-then-Aggregate, are proposed to produce factually grounded captions; the approach yields state-of-the-art reductions in object hallucination across multiple LVLMs and model sizes, with strong GPT-assisted evaluations. This work offers a practical, training-free path to mitigate hallucination by leveraging the model’s own verification capabilities and selective caption aggregation.

Abstract

Despite progress in Large Vision Language Models (LVLMs), object hallucination remains a critical issue in image captioning task, where models generate descriptions of non-existent objects, compromising their reliability. Previous work attributes this to LVLMs' over-reliance on language priors and attempts to mitigate it through logits calibration. However, they still lack a thorough analysis of the over-reliance. To gain a deeper understanding of over-reliance, we conduct a series of preliminary experiments, indicating that as the generation length increases, LVLMs' over-reliance on language priors leads to inflated probability of hallucinated object tokens, consequently exacerbating object hallucination. To circumvent this issue, we propose Language-Prior-Free Verification to enable LVLMs to faithfully verify the confidence of object existence. Based on this, we propose a novel training-free Self-Validation Framework to counter the over-reliance trap. It first validates objects' existence in sampled candidate captions and further mitigates object hallucination via caption selection or aggregation. Experiment results demonstrate that our framework mitigates object hallucination significantly in image captioning task (e.g., 65.6% improvement on CHAIRI metric with LLaVA-v1.5-7B), surpassing the previous SOTA methods. This result highlights a novel path towards mitigating hallucination by unlocking the inherent potential within LVLMs themselves.

Figures (13)

• Figure 1: Illustration of JSD trend: hallucinated objects are marked in red and existent objects in green. Within the same position span, object words often exhibit higher JSD values than other words. Early-position object words yield high JSD values, whereas words in later positions consistently show low JSD values.
• Figure 2: JSD value (top) and hallucination rate (bottom) for three methods across different relative position bins. As the generation progresses, JSD value decreases significantly, associated with a sharp increase in hallucination rate.
• Figure 3: Hallucinated objects are highlighted in red while existent objects are in green. The LVLM provides a discriminative existence estimation of the objects when Language-Prior-Free Verification is employed, indicating that LVLMs have the ability to self-validate their generated objects.
• Figure 4: Illustration of the Self-Validation Framework. The framework operates in two stages. Stage 1: An LVLM first generates multiple candidate captions. For each candidate, the framework extracts objects and employs LPFV to assess their confidence scores. Stage 2: The final caption is produced via one of two strategies: (a) Best-of-N Selection, where the candidate with the highest confidence is chosen, or (b) Filter-then-Aggregate, where sentences with low-confidence objects are discarded before aggregating the remaining content.
• Figure 5: Impact of different candidates nums $N$ on hallucination and recall performance.