Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey

Bhavuk Jain, Sercan Ö. Arık, Hardeo K. Thakur

Abstract

Multimodal large language models (MLLMs) integrate information from multiple modalities such as text, images, audio, and video, enabling complex capabilities such as visual question answering and audio translation. While powerful, this increased expressiveness introduces new and amplified vulnerabilities to adversarial manipulation. This survey provides a comprehensive and systematic analysis of adversarial threats to MLLMs, moving beyond enumerating attack techniques to explain the underlying causes of model susceptibility. We introduce a taxonomy that organizes adversarial attacks according to attacker objectives, unifying diverse attack surfaces across modalities and deployment settings. Additionally, we also present a vulnerability-centric analysis that links integrity attacks, safety and jailbreak failures, control and instruction hijacking, and training-time poisoning to shared architectural and representational weaknesses in multimodal systems. Together, this framework provides an explanatory foundation for understanding adversarial behavior in MLLMs and informs the development of more robust and secure multimodal language systems.

Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey

Abstract

Multimodal large language models (MLLMs) integrate information from multiple modalities such as text, images, audio, and video, enabling complex capabilities such as visual question answering and audio translation. While powerful, this increased expressiveness introduces new and amplified vulnerabilities to adversarial manipulation. This survey provides a comprehensive and systematic analysis of adversarial threats to MLLMs, moving beyond enumerating attack techniques to explain the underlying causes of model susceptibility. We introduce a taxonomy that organizes adversarial attacks according to attacker objectives, unifying diverse attack surfaces across modalities and deployment settings. Additionally, we also present a vulnerability-centric analysis that links integrity attacks, safety and jailbreak failures, control and instruction hijacking, and training-time poisoning to shared architectural and representational weaknesses in multimodal systems. Together, this framework provides an explanatory foundation for understanding adversarial behavior in MLLMs and informs the development of more robust and secure multimodal language systems.

Paper Structure

This paper contains 66 sections, 3 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background: Understanding LLMs in the Context of Adversarial Attacks
  3. LLM Architectures and Aspects Exploited by Attacks
  4. Mathematical Representation of LLMs and Adversarial Perturbations
  5. Taxonomy of Attacks on Multimodal LLMs
  6. A Framework for Classification
  7. Attack Taxonomy
  8. Integrity Attacks
  9. Signal Perturbations:
  10. Discrete Triggers:
  11. Representation and Fusion Exploits:
  12. Safety and Jailbreak Attacks
  13. Unimodal Surfaces:
  14. Multimodal Composite Jailbreaks:
  15. Universal Jailbreak Triggers:
  16. ...and 51 more sections

Figures (5)

  • Figure 1: A high-level overview of the adversarial attack landscape for Multimodal LLMs.
  • Figure 2: Hierarchical taxonomy of attacks on multimodal LLMs.
  • Figure 3: Representative attacks: visual perturbations, indirect prompt injection via images, visual safety bypasses, and inaudible audio commands.
  • Figure 4: A backdoor attack where the model is 'poisoned' during training.
  • Figure 5: Hierarchical Overview of LLM Vulnerabilities