MTRE: Multi-Token Reliability Estimation for Hallucination Detection in VLMs

TL;DR

MTRE addresses hallucination detection in vision-language models by moving beyond single-token signals to a multi-token, white-box reliability framework. It trains a token-level reliability classifier and aggregates per-token log-likelihood ratios across the first ten tokens, calibrated via cross-fitting and adaptive evidence length. Across MAD-Bench, MM-SafetyBench, MathVista, and arithmetic-geometry tasks, MTRE delivers significant gains in accuracy and AUROC, establishing state-of-the-art performance for open-source VLMs. The approach remains lightweight and scalable, offering practical contributions to safety and reliability in multimodal AI systems.

Abstract

Vision-language models (VLMs) now rival human performance on many multimodal tasks, yet they still hallucinate objects or generate unsafe text. Current hallucination detectors, e.g., single-token linear probing (LP) and PTrue, typically analyze only the logit of the first generated token or just its highest-scoring component, overlooking richer signals embedded within earlier token distributions. We demonstrate that analyzing the complete sequence of early logits potentially provides substantially more diagnostic information. We emphasize that hallucinations may only emerge after several tokens, as subtle inconsistencies accumulate over time. By analyzing the Kullback-Leibler (KL) divergence between logits corresponding to hallucinated and non-hallucinated tokens, we underscore the importance of incorporating later-token logits to more accurately capture the reliability dynamics of VLMs. In response, we introduce Multi-Token Reliability Estimation (MTRE), a lightweight, white-box method that aggregates logits from the first ten tokens using multi-token log-likelihood ratios and self-attention. Despite the challenges posed by large vocabulary sizes and long logit sequences, MTRE remains efficient and tractable. Across MAD-Bench, MM-SafetyBench, MathVista, and four compositional-geometry benchmarks, MTRE achieves a 9.4% gain in accuracy and a 14.8% gain in AUROC over standard detection methods, establishing a new state of the art in hallucination detection for open-source VLMs.

Figures (6)

• Figure 1: Summary of experiments on MAD-Bench and MM-Safety (5 methods on 2 detection tasks on 4 VLMs in 2 datasets): Each cell shows the fraction of experiments where the method in the row outperforms the method in the column measured by Accuracy and AUROC, respectively.
• Figure 2: We measure the KL divergence between the conditional probability distributions of the next token under hallucinated versus non-hallucinated generations, i.e., $\mathrm{softmax}(\ell_t)$ when $y_t$ is hallucinated versus $\mathrm{softmax}(\ell_t)$ when $y_t$ is non-hallucinated, in the Type 1 classification tasks and Type 2 self-evaluation tasks among different models and datasets.
• Figure 3: The KL divergence between hallucinated and non-hallucinated responses in the Arithmetic dataset (Type 1).
• Figure 4: Detection results on Type 1 Direct-answering task in MAD-Bench and MM-Safety-Bench. (For scores in table format see Appendix \ref{['Appendix:results']}, Tables \ref{['tab:oe_I_model_comparison']}, \ref{['tab:mq_I_model_comparison']}, and \ref{['tab:oeh_I_model_comparison']}).
• Figure 5: Detection results on Type 2 Direct-answering task in MAD-Bench and MM-Safety-Bench. (For scores in table format, see Appendix \ref{['Appendix:results']}, Tables \ref{['tab:oe_II_model_comparison']}, \ref{['tab:mq_II_model_comparison']}, and \ref{['tab:oeh_II_model_comparison']} ).