HARMONY: Hidden Activation Representations and Model Output-Aware Uncertainty Estimation for Vision-Language Models

TL;DR

This work tackles uncertainty estimation for Vision-Language Models by addressing inter-token dependencies and language-prior biases. It introduces HARMONY, a scoring framework that fuses hidden activation representations, per-token probabilities, and generated text using a VisualBERT-based input mapping to capture multimodal alignment at token level. Empirical results across A-OKVQA, VizWiz, and PathVQA with multiple VLMs show consistent gains in UE (AUROC/PRR) and selective prediction performance, along with ablations highlighting the importance of multi-signal inputs and layer selection. The method also demonstrates robustness under distribution shifts and provides guidance on calibration data requirements, enabling more reliable and safe model outputs in practical VLM deployments.

Abstract

Uncertainty Estimation (UE) plays a central role in quantifying the reliability of model outputs and reducing unsafe generations via selective prediction. In this regard, most existing probability-based UE approaches rely on predefined functions, aggregating token probabilities into a single UE score using heuristics such as length-normalization. However, these methods often fail to capture the complex relationships between generated tokens and struggle to identify biased probabilities often influenced by \textbf{language priors}. Another line of research uses hidden representations of the model and trains simple MLP architectures to predict uncertainty. However, such functions often lose the intricate \textbf{ inter-token dependencies}. While prior works show that hidden representations encode multimodal alignment signals, our work demonstrates that how these signals are processed has a significant impact on the UE performance. To effectively leverage these signals to identify inter-token dependencies, and vision-text alignment, we propose \textbf{HARMONY} (Hidden Activation Representations and Model Output-Aware Uncertainty Estimation for Vision-Language Models), a novel UE framework that integrates generated tokens ('text'), model's uncertainty score at the output ('MaxProb'), and its internal belief on the visual understanding of the image and the generated token (captured by 'hidden representations') at token level via appropriate input mapping design and suitable architecture choice. Our experimental experiments across two open-ended VQA benchmarks (A-OKVQA, and VizWiz) and four state-of-the-art VLMs (LLaVA-7B, LLaVA-13B, InstructBLIP, and Qwen-VL) show that HARMONY consistently matches or surpasses existing approaches, achieving up to 5\% improvement in AUROC and 9\% in PRR.

Figures (7)

• Figure 1: Illustration of VLM inference and the proposed framework, HARMONY. VLMs as autoregressive model take image and question tokens as input and generate text tokens sequentially. During generation, certain tokens (e.g., "up" in the example) carry higher semantic relevance to the input, reflecting the inherently complex inter-token dependencies in VLM outputs. Moreover, some responses may be weakly grounded in the image and instead dominated by language priors, making UE particularly challenging. To address these issues, our proposed framework, HARMONY, jointly leverages the model’s hidden representations as a vision-text alignment signal together with the generated tokens and their sequential token-level probabilities, producing a single UE score for the generated answer.
• Figure 2: Illustration of language priors challenge where the LLaVa-7B model generates incorrect outputs with high confidence on visually uncertain (unanswerable) questions from the VizWiz dataset.
• Figure 3: An illustration of selective prediction decisions on the A-OKVQA dataset with LLaVa-7b model. In the left-most example, the model generates an incorrect answer, yet both LARS yaldiz2024not MSF whitehead2022reliable choose to answer/predict based on their respective calibration thresholds. In the second example, LARS opts to answer, while MSF correctly abstains. In the right-most example, both methods abstain, whereas HARMONY makes the right prediction for each of these examples.
• Figure 4: Out-of-Distribution Generalization.
• Figure 5: Effect of calibration data size on the UE performance.