When Vision Fails: Text Attacks Against ViT and OCR

TL;DR

This work reveals a new vulnerability where Vision Transformers and OCR systems can be misled by Unicode-based adversarial inputs encoded in text but rendered as images. It introduces a black-box, gradient-free differential-evolution attack that injects combining diacritics, producing small visual perturbations that degrade downstream tasks such as translation and toxicity detection, even on production models from major vendors. A human-user study confirms that these perturbations do not hinder readability or comprehension, highlighting a gap between human perceptual robustness and machine sensitivity. The findings motivate defenses at the rendering stage, notably removing combining diacritics or integrating rendering-aware processing, to mitigate these Unicode-driven attacks in practical systems.

Abstract

Text-based machine learning models are vulnerable to an emerging class of Unicode-based adversarial examples capable of tricking a model into misreading text with potentially disastrous effects. The primary existing defense against these attacks is to preprocess potentially malicious text inputs using optical character recognition (OCR). In theory, OCR models will ignore any malicious Unicode characters and will extract the visually correct input to be fed to the model. In this work, we show that these visual defenses fail to prevent this type of attack. We use a genetic algorithm to generate visual adversarial examples (i.e., OCR outputs) in a black-box setting, demonstrating a highly effective novel attack that substantially reduces the accuracy of OCR and other visual models. Specifically, we use the Unicode functionality of combining characters (e.g., ñ which combines the characters n and ~) to manipulate text inputs so that small visual perturbations appear when the text is displayed. We demonstrate the effectiveness of these attacks in the real world by creating adversarial examples against production models published by Meta, Microsoft, IBM, and Google. We additionally conduct a user study to establish that the model-fooling adversarial examples do not affect human comprehension of the text, showing that language models are uniquely vulnerable to this type of text attack.

Figures (11)

• Figure 1: The visualization gap, against which visual models claim to be robust, continues to be a source of adversarial examples via diacritics as in this toxic content classifier example.
• Figure 2: Our attack: adversarial examples in the visual domain encoded in the textual domain.
• Figure 3: Performance of evaluated models across perturbation budgets relative to no perturbations.
• Figure 4: Evaluation of diacritic adversarial examples against TrOCR.
• Figure 5: Eval of diacritic adv. examples against FairSeq EN->FR translation defended by TrOCR.