MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification

TL;DR

MetaToken introduces a lightweight token-level hallucination detector for LVLM outputs based on meta-classification. By extracting a broad set of features from model outputs and training a small binary classifier, it can detect hallucinated objects without ground-truth data and with calibrated uncertainty, achieving high AUROC and AUPRC across four open-source LVLMs. The approach demonstrates strong integration potential with existing mitigation methods (e.g., LURE) and offers practical gains in reducing hallucinations while maintaining low computational overhead. This work provides interpretable insights into the factors driving LVLM hallucinations and delivers a reproducible framework for token-level evaluation and detection.

Abstract

Large Vision Language Models (LVLMs) have shown remarkable capabilities in multimodal tasks like visual question answering or image captioning. However, inconsistencies between the visual information and the generated text, a phenomenon referred to as hallucinations, remain an unsolved problem with regard to the trustworthiness of LVLMs. To address this problem, recent works proposed to incorporate computationally costly Large (Vision) Language Models in order to detect hallucinations on a sentence- or subsentence-level. In this work, we introduce MetaToken, a lightweight binary classifier to detect hallucinations on the token-level at negligible cost. Based on a statistical analysis, we reveal key factors of hallucinations in LVLMs. MetaToken can be applied to any open-source LVLM without any knowledge about ground truth data providing a calibrated detection of hallucinations. We evaluate our method on four state-of-the-art LVLMs demonstrating the effectiveness of our approach.

Figures (9)

• Figure 1: MetaToken. Based on generated image captions (i), we calculate our proposed input features (ii) (see \ref{['subsec:metrics']}). Afterwards, we apply the trained meta classifier (iii) to detect hallucinated and true objects (iv). Moreover, (v) MetaToken can be easily integrated into LURE Zhou.2023 to improve the hallucination mitigation.
• Figure 2: Probability Features. Visualization of the probability features log probability $L$ (\ref{['eq:logp']}), variance $V$ (\ref{['eq:var']}) and entropy $E$ (\ref{['eq:entropy']}) for true and hallucinated objects. (beam) and (sample) reflect the beam search and nucleus sampling algorithm, respectively.
• Figure 3: LASSO Path. LASSO path for $\mathcal{M}$. $A$ denotes the maximum of the absolute values of all $G$ weight coefficients for the attention features $A^{g}, g=0,\dots,G-1$.
• Figure 4: $AUPRC$ as a Function of the Number of Features. The classification performance of MetaToken in terms of $AUPRC$ as a function of the number of features for different LVLMs. The features are selected along the LASSO path of the respective LVLM.
• Figure 5: LASSO Path based on Nucleus Sampling Holtzman.2020. LASSO path for $\mathcal{M}$ (\ref{['eq:M']}). $A$ denotes the maximum of the absolute values of all $G$ weight coefficients for the attention features $A^{g}, g=0,\dots,G-1$.