Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security

Yihe Fan, Yuxin Cao, Ziyu Zhao, Ziyao Liu, Shaofeng Li

TL;DR

This survey addresses the security challenges introduced by image inputs in multimodal large language models (MLLMs). It constructs a threat model, catalogues structure-, perturbation-, and data-poisoning-based attacks, and reviews corresponding training-time and inference-time defenses, highlighting gaps in cross-modal security alignment and privacy protection. Key contributions include a taxonomy of vulnerabilities and attack objectives (e.g., jailbreak, prompt injection, backdoors) and proposed directions such as robust prompts, NL feedback alignment, and privacy-enhancing techniques. The work underscores the practical importance of developing trustworthy MLLMs as these models become increasingly integrated into safety-critical and everyday applications, and it motivates future standards for evaluating and mitigating multimodal security risks.

Abstract

Multimodal Large Language Models (MLLMs) demonstrate remarkable capabilities that increasingly influence various aspects of our daily lives, constantly defining the new boundary of Artificial General Intelligence (AGI). Image modalities, enriched with profound semantic information and a more continuous mathematical nature compared to other modalities, greatly enhance the functionalities of MLLMs when integrated. However, this integration serves as a double-edged sword, providing attackers with expansive vulnerabilities to exploit for highly covert and harmful attacks. The pursuit of reliable AI systems like powerful MLLMs has emerged as a pivotal area of contemporary research. In this paper, we endeavor to demostrate the multifaceted risks associated with the incorporation of image modalities into MLLMs. Initially, we delineate the foundational components and training processes of MLLMs. Subsequently, we construct a threat model, outlining the security vulnerabilities intrinsic to MLLMs. Moreover, we analyze and summarize existing scholarly discourses on MLLMs' attack and defense mechanisms, culminating in suggestions for the future research on MLLM security. Through this comprehensive analysis, we aim to deepen the academic understanding of MLLM security challenges and propel forward the development of trustworthy MLLM systems.

Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security

TL;DR

This survey addresses the security challenges introduced by image inputs in multimodal large language models (MLLMs). It constructs a threat model, catalogues structure-, perturbation-, and data-poisoning-based attacks, and reviews corresponding training-time and inference-time defenses, highlighting gaps in cross-modal security alignment and privacy protection. Key contributions include a taxonomy of vulnerabilities and attack objectives (e.g., jailbreak, prompt injection, backdoors) and proposed directions such as robust prompts, NL feedback alignment, and privacy-enhancing techniques. The work underscores the practical importance of developing trustworthy MLLMs as these models become increasingly integrated into safety-critical and everyday applications, and it motivates future standards for evaluating and mitigating multimodal security risks.

Abstract

Multimodal Large Language Models (MLLMs) demonstrate remarkable capabilities that increasingly influence various aspects of our daily lives, constantly defining the new boundary of Artificial General Intelligence (AGI). Image modalities, enriched with profound semantic information and a more continuous mathematical nature compared to other modalities, greatly enhance the functionalities of MLLMs when integrated. However, this integration serves as a double-edged sword, providing attackers with expansive vulnerabilities to exploit for highly covert and harmful attacks. The pursuit of reliable AI systems like powerful MLLMs has emerged as a pivotal area of contemporary research. In this paper, we endeavor to demostrate the multifaceted risks associated with the incorporation of image modalities into MLLMs. Initially, we delineate the foundational components and training processes of MLLMs. Subsequently, we construct a threat model, outlining the security vulnerabilities intrinsic to MLLMs. Moreover, we analyze and summarize existing scholarly discourses on MLLMs' attack and defense mechanisms, culminating in suggestions for the future research on MLLM security. Through this comprehensive analysis, we aim to deepen the academic understanding of MLLM security challenges and propel forward the development of trustworthy MLLM systems.
Paper Structure (39 sections, 1 figure, 2 tables)

This paper contains 39 sections, 1 figure, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. BACKGROUND
  3. Model Structure
  4. Modality Encoders
  5. Input Projector
  6. LLM Backbone
  7. Output Projector
  8. Modality Generators
  9. Training Process
  10. Multimodal Pre-training
  11. Multimodal Instruction Tuning
  12. THREAT MODEL
  13. Vulnerabilities
  14. Training Datasets
  15. Multimodal Input
  16. ...and 24 more sections

Figures (1)

  • Figure 1: The general model architecture of MLLMs and the vulnerabilities that attackers may exploit to manipulate the model to generate malicious output.