Attention! Your Vision Language Model Could Be Maliciously Manipulated

TL;DR

This work reveals a fundamental vulnerability of vision-language models to imperceptible visual perturbations and presents VMA, a unified attack framework that precisely manipulates output tokens. By combining a differentiable input transformation with momentum-based optimization, VMA demonstrates high effectiveness across jailbreaking, hijacking, hallucination, privacy breaches, Denial-of-Service, sponge examples, and watermarking, under white-box conditions. The authors provide theoretical guarantees via certified radii showing visual perturbations can have greater impact than textual changes, and validate results across eight tasks and multiple models with strong empirical evidence. The findings highlight urgent needs for robust defenses and more secure training/ deployment practices for VLMs, while also suggesting potential benign applications like watermarking; however, transferability across architectures remains a limitation.

Abstract

Large Vision-Language Models (VLMs) have achieved remarkable success in understanding complex real-world scenarios and supporting data-driven decision-making processes. However, VLMs exhibit significant vulnerability against adversarial examples, either text or image, which can lead to various adversarial outcomes, e.g., jailbreaking, hijacking, and hallucination, etc. In this work, we empirically and theoretically demonstrate that VLMs are particularly susceptible to image-based adversarial examples, where imperceptible perturbations can precisely manipulate each output token. To this end, we propose a novel attack called Vision-language model Manipulation Attack (VMA), which integrates first-order and second-order momentum optimization techniques with a differentiable transformation mechanism to effectively optimize the adversarial perturbation. Notably, VMA can be a double-edged sword: it can be leveraged to implement various attacks, such as jailbreaking, hijacking, privacy breaches, Denial-of-Service, and the generation of sponge examples, etc, while simultaneously enabling the injection of watermarks for copyright protection. Extensive empirical evaluations substantiate the efficacy and generalizability of VMA across diverse scenarios and datasets. Code is available at https://github.com/Trustworthy-AI-Group/VMA.

Figures (4)

• Figure 1: Overview of the adversarial attacks for VLMs and the architecture of mainstream VLMs. (a) Typical attacks append an adversarial suffix to the prompt or inject text into image for jailbreaking. In contrast, VMA applies an imperceptible perturbation to precisely manipulate the output while maintaining visual fidelity, enabling numerous attacks. (b) The architecture of mainstream VLMs adopts an LLM decoder. (c) The computation process of attention, which enables multimodal fusion.
• Figure 2: Average token-wise distribution change on textual and visual perturbation. Visual perturbation has a greater effect than textual perturbation on the output probability.
• Figure 3: Adversarial images generated by the proposed VMA to manipulate various VLMs to output two specific sequences, namely "Attention! You vision language model could be maliciously manipulated." (the upper row) and "The Thirty-Ninth Annual Conference on Neural Information Processing Systems." (the lower row) with the prompt "Please describe the image."
• Figure 4: Average inference time (s) and ASR (%) of VMA for Sponge Examples attack across four VLMs.

Theorems & Definitions (8)

• Definition 1: Certified Radius
• Theorem 1
• Theorem 2
• Corollary 1
• Definition 2
• Lemma 1
• proof
• proof