Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ImgTrojan: Jailbreaking Vision-Language Models with ONE Image

Xijia Tao, Shuai Zhong, Lei Li, Qi Liu, Lingpeng Kong

TL;DR

<3-5 sentence high-level summary> ImgTrojan exposes a cross-modal data-poisoning vulnerability in vision-language models by embedding jailbreak prompts into a small set of image-caption pairs during training, enabling clean images to trigger harmful responses at inference via an image-to-JBP association. The attack achieves high Attack Success Rates (ASR) with minimal degradation to caption quality, and persists even after subsequent fine-tuning with clean data, highlighting weaknesses in data-source pipelines and post-training alignment. The study analyzes poison ratios, identifies the Trojan’s locus primarily in the LLM component, and demonstrates transferability across model families (e.g., LLaVA and Qwen-VL), while showing that standard filtering and some defensive strategies offer limited protection. These findings stress the urgency of developing robust data-sourcing defenses and safer instruction-tuning practices for open-source VLM ecosystems.

Abstract

There has been an increasing interest in the alignment of large language models (LLMs) with human values. However, the safety issues of their integration with a vision module, or vision language models (VLMs), remain relatively underexplored. In this paper, we propose a novel jailbreaking attack against VLMs, aiming to bypass their safety barrier when a user inputs harmful instructions. A scenario where our poisoned (image, text) data pairs are included in the training data is assumed. By replacing the original textual captions with malicious jailbreak prompts, our method can perform jailbreak attacks with the poisoned images. Moreover, we analyze the effect of poison ratios and positions of trainable parameters on our attack's success rate. For evaluation, we design two metrics to quantify the success rate and the stealthiness of our attack. Together with a list of curated harmful instructions, a benchmark for measuring attack efficacy is provided. We demonstrate the efficacy of our attack by comparing it with baseline methods.

ImgTrojan: Jailbreaking Vision-Language Models with ONE Image

TL;DR

<3-5 sentence high-level summary> ImgTrojan exposes a cross-modal data-poisoning vulnerability in vision-language models by embedding jailbreak prompts into a small set of image-caption pairs during training, enabling clean images to trigger harmful responses at inference via an image-to-JBP association. The attack achieves high Attack Success Rates (ASR) with minimal degradation to caption quality, and persists even after subsequent fine-tuning with clean data, highlighting weaknesses in data-source pipelines and post-training alignment. The study analyzes poison ratios, identifies the Trojan’s locus primarily in the LLM component, and demonstrates transferability across model families (e.g., LLaVA and Qwen-VL), while showing that standard filtering and some defensive strategies offer limited protection. These findings stress the urgency of developing robust data-sourcing defenses and safer instruction-tuning practices for open-source VLM ecosystems.

Abstract

There has been an increasing interest in the alignment of large language models (LLMs) with human values. However, the safety issues of their integration with a vision module, or vision language models (VLMs), remain relatively underexplored. In this paper, we propose a novel jailbreaking attack against VLMs, aiming to bypass their safety barrier when a user inputs harmful instructions. A scenario where our poisoned (image, text) data pairs are included in the training data is assumed. By replacing the original textual captions with malicious jailbreak prompts, our method can perform jailbreak attacks with the poisoned images. Moreover, we analyze the effect of poison ratios and positions of trainable parameters on our attack's success rate. For evaluation, we design two metrics to quantify the success rate and the stealthiness of our attack. Together with a list of curated harmful instructions, a benchmark for measuring attack efficacy is provided. We demonstrate the efficacy of our attack by comparing it with baseline methods.
Paper Structure (42 sections, 6 figures, 14 tables)

This paper contains 42 sections, 6 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Vision-Language Models (VLMs)
  4. Explorations of Jailbreaking
  5. Methods
  6. Task Formulation
  7. ImgTrojan: Clean Images as Trojan
  8. Jailbreaking Evaluation
  9. Clean Metric
  10. Attack Success Rate
  11. Downstream VQA Performance
  12. Experiments
  13. Experimental Setup
  14. Target Models
  15. Training Data
  16. ...and 27 more sections

Figures (6)

  • Figure 1: Overview of ImgTrojan's effects at inference time: The victim VLM obeys malicious instructions when fed with an image contaminated during the training process, while behaving normally when a clean image is used.
  • Figure 2: The flowchart depicting the ImgTrojan jailbreaking process. JBP is injected into the captions of images during SFT.
  • Figure 3: ASR (illustrated as bar plots) and clean metric (illustrated as line plots) results for ImgTrojan attack: (a)–(b) with different poison ratios on LLaVA-v1.5 7B, and (c)–(d) with a fixed poison ratio of 0.001 on models with different sizes, both with the settings of two different JBPs used for poisoning and two prompting methods.
  • Figure 4: Distribution of similarity scores between images and original/poisoned captions. There are 78.07% of poisoned caption-image pairs that could pass the 0.3 similarity threshold.
  • Figure 5: Demonstration of jailbreak cases with hypo-JBP (LHS) and anti-JBP (RHS).
  • ...and 1 more figures