Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Analysing the Robustness of Vision-Language-Models to Common Corruptions

Muhammad Usama, Syeda Aishah Asim, Syed Bilal Ali, Syed Talal Wasim, Umair Bin Mansoor

TL;DR

This study analyzes the robustness of vision-language models to common image corruptions by evaluating 19 ImageNet-C types across noise, blur, weather, and digital categories. It introduces two corruption-augmented benchmarks, TextVQA-C and GQA-C, and uses the LLaVA-1.5 model to uncover task-specific vulnerability patterns. A frequency-domain perspective shows that transformers’ low-frequency processing bias explains why text understanding and object reasoning degrade differently under various corruptions. The findings provide practical guidance for designing corruption-robust vision-language architectures suitable for real-world deployment, particularly in OCR-enabled text reasoning and scene understanding tasks.

Abstract

Vision-language models (VLMs) have demonstrated impressive capabilities in understanding and reasoning about visual and textual content. However, their robustness to common image corruptions remains under-explored. In this work, we present the first comprehensive analysis of VLM robustness across 19 corruption types from the ImageNet-C benchmark, spanning four categories: noise, blur, weather, and digital distortions. We introduce two new benchmarks, TextVQA-C and GQA-C, to systematically evaluate how corruptions affect scene text understanding and object-based reasoning, respectively. Our analysis reveals that transformer-based VLMs exhibit distinct vulnerability patterns across tasks: text recognition deteriorates most severely under blur and snow corruptions, while object reasoning shows higher sensitivity to corruptions such as frost and impulse noise. We connect these observations to the frequency-domain characteristics of different corruptions, revealing how transformers' inherent bias toward low-frequency processing explains their differential robustness patterns. Our findings provide valuable insights for developing more corruption-robust vision-language models for real-world applications.

Analysing the Robustness of Vision-Language-Models to Common Corruptions

TL;DR

This study analyzes the robustness of vision-language models to common image corruptions by evaluating 19 ImageNet-C types across noise, blur, weather, and digital categories. It introduces two corruption-augmented benchmarks, TextVQA-C and GQA-C, and uses the LLaVA-1.5 model to uncover task-specific vulnerability patterns. A frequency-domain perspective shows that transformers’ low-frequency processing bias explains why text understanding and object reasoning degrade differently under various corruptions. The findings provide practical guidance for designing corruption-robust vision-language architectures suitable for real-world deployment, particularly in OCR-enabled text reasoning and scene understanding tasks.

Abstract

Vision-language models (VLMs) have demonstrated impressive capabilities in understanding and reasoning about visual and textual content. However, their robustness to common image corruptions remains under-explored. In this work, we present the first comprehensive analysis of VLM robustness across 19 corruption types from the ImageNet-C benchmark, spanning four categories: noise, blur, weather, and digital distortions. We introduce two new benchmarks, TextVQA-C and GQA-C, to systematically evaluate how corruptions affect scene text understanding and object-based reasoning, respectively. Our analysis reveals that transformer-based VLMs exhibit distinct vulnerability patterns across tasks: text recognition deteriorates most severely under blur and snow corruptions, while object reasoning shows higher sensitivity to corruptions such as frost and impulse noise. We connect these observations to the frequency-domain characteristics of different corruptions, revealing how transformers' inherent bias toward low-frequency processing explains their differential robustness patterns. Our findings provide valuable insights for developing more corruption-robust vision-language models for real-world applications.

Paper Structure

This paper contains 23 sections, 4 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contrastive Vision Language Encoders
  4. Evolution of Large Language Models
  5. Modern Vision-Language Models
  6. Robustness Challenges in Vision-Language Systems
  7. Preliminaries: Common Corruptions
  8. Noise Corruptions
  9. Blur Corruptions
  10. Weather Corruptions
  11. Digital Corruptions
  12. Impact of Severity Levels
  13. Evaluation Methodology
  14. LLaVA 1.5 Vision Language Model
  15. Corruption-Augmented Benchmark Datasets
  16. ...and 8 more sections

Figures (7)

  • Figure 1: Different noise types across severity levels
  • Figure 2: Overview of the LLaVA architecture
  • Figure 3: Sample image-text pairs from TextVQA and GQA.
  • Figure 4: Accuracy Results for the Noise Corruption Category
  • Figure 5: Accuracy Results for the Blur Corruption Category
  • ...and 2 more figures