Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantification
Tao Huang, Rui Wang, Xiaofei Liu, Yi Qin, Li Duan, Liping Jing
TL;DR
This paper tackles misbehaviors in large vision-language models by treating epistemic uncertainty as two distinct components: internal information conflict and ignorance due to missing information. It introduces Evidential Uncertainty Quantification (EUQ), which maps pre-logits from the LVLM output head to evidence and uses Dempster–Shafer theory to compute conflict and ignorance in a single forward pass, without training. Across hallucinations, jailbreaks, adversarial attacks, and OOD failures, EUQ demonstrates consistent improvements over strong baselines in AUROC and AUPR, and reveals interpretable layer-wise dynamics where ignorance shrinks while conflict grows deeper in the decoder. The method is efficient, scalable across model sizes, and generalizable to any architecture with a linear projector, offering a practical tool for misbehavior detection and model interpretation.
Abstract
Large vision-language models (LVLMs) have shown substantial advances in multimodal understanding and generation. However, when presented with incompetent or adversarial inputs, they frequently produce unreliable or even harmful content, such as fact hallucinations or dangerous instructions. This misalignment with human expectations, referred to as \emph{misbehaviors} of LVLMs, raises serious concerns for deployment in critical applications. These misbehaviors are found to stem from epistemic uncertainty, specifically either conflicting internal knowledge or the absence of supporting information. However, existing uncertainty quantification methods, which typically capture only overall epistemic uncertainty, have shown limited effectiveness in identifying such issues. To address this gap, we propose Evidential Uncertainty Quantification (EUQ), a fine-grained method that captures both information conflict and ignorance for effective detection of LVLM misbehaviors. In particular, we interpret features from the model output head as either supporting (positive) or opposing (negative) evidence. Leveraging Evidence Theory, we model and aggregate this evidence to quantify internal conflict and knowledge gaps within a single forward pass. We extensively evaluate our method across four categories of misbehavior, including hallucinations, jailbreaks, adversarial vulnerabilities, and out-of-distribution (OOD) failures, using state-of-the-art LVLMs, and find that EUQ consistently outperforms strong baselines, showing that hallucinations correspond to high internal conflict and OOD failures to high ignorance. Furthermore, layer-wise evidential uncertainty dynamics analysis helps interpret the evolution of internal representations from a new perspective. The source code is available at https://github.com/HT86159/EUQ.