VRSA: Jailbreaking Multimodal Large Language Models through Visual Reasoning Sequential Attack

TL;DR

The paper presents VRSA, a novel framework that jailbreaks multimodal LLMs by transforming harmful intents into sequential visual reasoning tasks. It introduces Adaptive Scene Refinement to ensure scene plausibility, Semantic Coherent Completion to maintain cross-subtext coherence, and Text-Image Consistency Alignment to preserve semantic alignment between text and generated images. Through extensive experiments on open- and closed-source MLLMs, VRSA achieves superior attack success rates and toxicity scores compared with prior methods, validating the importance of scene-aware, sequential visual reasoning in safety evaluation. The work provides a rigorous methodology and prompts for assessing visual reasoning vulnerabilities, with code released for reproducibility and further research.

Abstract

Multimodal Large Language Models (MLLMs) are widely used in various fields due to their powerful cross-modal comprehension and generation capabilities. However, more modalities bring more vulnerabilities to being utilized for jailbreak attacks, which induces MLLMs to output harmful content. Due to the strong reasoning ability of MLLMs, previous jailbreak attacks try to explore reasoning safety risk in text modal, while similar threats have been largely overlooked in the visual modal. To fully evaluate potential safety risks in the visual reasoning task, we propose Visual Reasoning Sequential Attack (VRSA), which induces MLLMs to gradually externalize and aggregate complete harmful intent by decomposing the original harmful text into several sequentially related sub-images. In particular, to enhance the rationality of the scene in the image sequence, we propose Adaptive Scene Refinement to optimize the scene most relevant to the original harmful query. To ensure the semantic continuity of the generated image, we propose Semantic Coherent Completion to iteratively rewrite each sub-text combined with contextual information in this scene. In addition, we propose Text-Image Consistency Alignment to keep the semantical consistency. A series of experiments demonstrates that the VRSA can achieve a higher attack success rate compared with the state-of-the-art jailbreak attack methods on both the open-source and closed-source MLLMs such as GPT-4o and Claude-4.5-Sonnet.

Figures (17)

• Figure 1: Illustration of Visual Reasoning Sequential Attack(VRSA). Our VRSA generates the image sequences based on original harmful intentions, and combine with the pre-defined text prompt to guide the visual reasoning process of MLLMs in generating harmful contexts, which can evaluate the potential risks of MLLMs in visual reasoning tasks.
• Figure 2: Framework of our VRSA. Based on a harmful text $T$, VRSA operates in four stages: (1) A scene judge model selects and rewrites the scene $\mathcal{S}$ most relevant to the harmful text $T$, and decomposes it into scene-grounded sub-texts. (2) A semantic judge model and a revise model mask and reconstruct the semantically low-relevance content within the sub-texts. (3) A Text-to-Image model iteratively generates sub-images that are semantically aligned with the sub-texts, and the semantic similarity is measured by Visual-language Model (CLIP). (4) The sub-images are combined with a pre-defined guided texts to jailbreak the MLLMs.
• Figure 3: An example of Adaptive Scene Refinement. The left panel shows the initial scene retrieved from the Scene library, and the right panel shows the final optimized scene after refinement, which is more closely related to the original harmful intention.
• Figure 4: Ablation on the number of images in visual reasoning sequence. The horizontal axis represents the number of images in the visual reasoning sequence, while the two curves illustrate the trends of ASR and Toxic scores as the number of images increases.
• Figure 5: Ablation on the maximum iterations $N_{1}$ of adaptive scene refinement. The horizontal axis represents the num of maximum iterations, while the two curves illustrate the trends of ASR and Toxic score as the num of maximum iterations increases.