Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Are vision language models robust to uncertain inputs?

Xi Wang, Eric Nalisnick

TL;DR

The paper investigates whether vision-language models (VLMs) are robust to uncertain inputs, challenging the notion that simply scaling model size cures uncertainty. Through anomaly detection and classification-with-rejection experiments across ImageNet-C, CIFAR-10 mapped tasks, ECG data, and Galaxy Zoo images, it shows that larger, more recent VLMs improve robustness but can still produce confident yet incorrect outputs when inputs are unclear. A key finding is that prompting models to abstain from uncertain predictions substantially enhances reliability on natural images, while domain-specific tasks reveal gaps without domain knowledge. The authors further propose caption diversity as a label-free proxy for internal uncertainty, enabling prediction of abstention behavior without ground-truth labels, and discuss limitations and future directions, including domain-knowledge augmentation for specialized tasks.

Abstract

Robustness against uncertain and ambiguous inputs is a critical challenge for deep learning models. While recent advancements in large scale vision language models (VLMs, e.g. GPT4o) might suggest that increasing model and training dataset size would mitigate this issue, our empirical evaluation shows a more complicated picture. Testing models using two classic uncertainty quantification tasks, anomaly detection and classification under inherently ambiguous conditions, we find that newer and larger VLMs indeed exhibit improved robustness compared to earlier models, but still suffer from a tendency to strictly follow instructions, often causing them to hallucinate confident responses even when faced with unclear or anomalous inputs. Remarkably, for natural images such as ImageNet, this limitation can be overcome without pipeline modifications: simply prompting models to abstain from uncertain predictions enables significant reliability gains, achieving near-perfect robustness in several settings. However, for domain-specific tasks such as galaxy morphology classification, a lack of specialized knowledge prevents reliable uncertainty estimation. Finally, we propose a novel mechanism based on caption diversity to reveal a model's internal uncertainty, enabling practitioners to predict when models will successfully abstain without relying on labeled data.

Are vision language models robust to uncertain inputs?

TL;DR

The paper investigates whether vision-language models (VLMs) are robust to uncertain inputs, challenging the notion that simply scaling model size cures uncertainty. Through anomaly detection and classification-with-rejection experiments across ImageNet-C, CIFAR-10 mapped tasks, ECG data, and Galaxy Zoo images, it shows that larger, more recent VLMs improve robustness but can still produce confident yet incorrect outputs when inputs are unclear. A key finding is that prompting models to abstain from uncertain predictions substantially enhances reliability on natural images, while domain-specific tasks reveal gaps without domain knowledge. The authors further propose caption diversity as a label-free proxy for internal uncertainty, enabling prediction of abstention behavior without ground-truth labels, and discuss limitations and future directions, including domain-knowledge augmentation for specialized tasks.

Abstract

Robustness against uncertain and ambiguous inputs is a critical challenge for deep learning models. While recent advancements in large scale vision language models (VLMs, e.g. GPT4o) might suggest that increasing model and training dataset size would mitigate this issue, our empirical evaluation shows a more complicated picture. Testing models using two classic uncertainty quantification tasks, anomaly detection and classification under inherently ambiguous conditions, we find that newer and larger VLMs indeed exhibit improved robustness compared to earlier models, but still suffer from a tendency to strictly follow instructions, often causing them to hallucinate confident responses even when faced with unclear or anomalous inputs. Remarkably, for natural images such as ImageNet, this limitation can be overcome without pipeline modifications: simply prompting models to abstain from uncertain predictions enables significant reliability gains, achieving near-perfect robustness in several settings. However, for domain-specific tasks such as galaxy morphology classification, a lack of specialized knowledge prevents reliable uncertainty estimation. Finally, we propose a novel mechanism based on caption diversity to reveal a model's internal uncertainty, enabling practitioners to predict when models will successfully abstain without relying on labeled data.
Paper Structure (23 sections, 9 figures, 3 tables)

This paper contains 23 sections, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background and related work
  3. Improving models' robustness against uncertain inputs
  4. Uncertainty quantification in large language models
  5. Uncertainty quantification in vision language models
  6. Methods
  7. Evaluation of VLM's implicit uncertainty quantification ability
  8. Caption diversity for understanding the underlying mechanism of rejection
  9. Experiments
  10. Anomaly detection
  11. Classification with rejection under ambiguous inputs
  12. Conclusion
  13. Limitation
  14. Prompt
  15. Anomaly detection
  16. ...and 8 more sections

Figures (9)

  • Figure 1: Classic uncertainty quantification tasks revisited in the VLMs era. Using CIFAR-10 as an example, we illustrate how corrupted inputs and inputs from outside CIFAR-10 concepts expose different challenges and failure modes in small supervised models vs. large vision language models (VLMs, e.g. GPT4o) prompted to do classification, despite sharing the same evaluation data.
  • Figure 2: VLMs show degraded performance under corrupted inputs, allowing rejection helps maintain reliability.Top row demonstrates VLMs' outputs for a selected sample with or without a rejection prompt appended. Bottom row shows the classification accuracy under standard prompt without (solid line) vs. with (dashed line) rejection instruction prompt appended.
  • Figure 3: Enabling the rejection option allows VLM to pick out anomaly inputs, preventing hallucination. When prompted to classify a random line image into normal v.s. abnormal ECG signal, certain VLM (Qwen 2.5 3B) would generate hallucinated results (left), potentially caused by its tendency towards strictly following instructions sharma2023towards. However, when the prompt explicitly permits rejecting non-ECG input (additional red texts in the prompt), the same model correctly identifies the input as anomalous and responds with Unknown (right).
  • Figure 4: VLMs generate diverse captions for ambiguous images. We prompt Qwen2.5 7B to "generate a description" given an input image under different random seeds. Clean image from ImageNet receives consistent captions while its corrupted version having a diverse set of captions (top left v.s. bottom left). For the galaxy image where annotators show significant disagreement on whether there exist spiral arms (bottom right), VLMs fail to have diversity in the caption, indicating that the model does not understand the ambiguity, likely due to limited domain knowledge.
  • Figure 5: Caption diversity reflects model uncertainty under ambiguous inputs. We empirically verified the hypothesis from Sec. \ref{['sec:caption_diversity']}. Top: As corruption increases, caption diversity rises across all models, indicating greater uncertainty. Bottom:Rejected samples exhibit higher caption diversity than classified ones, suggesting that diversity of independently-generated captions correlates with models' internal uncertainty level for an input and the tendency for rejecting it when prompted.
  • ...and 4 more figures