Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigm

Ye Leng, Junjie Chu, Mingjie Li, Chenhao Lin, Chao Shen, Michael Backes, Yun Shen, Yang Zhang

Abstract

Recently, multimodal large language models (MLLMs) have emerged as a unified paradigm for language and image generation. Compared with diffusion models, MLLMs possess a much stronger capability for semantic understanding, enabling them to process more complex textual inputs and comprehend richer contextual meanings. However, this enhanced semantic ability may also introduce new and potentially greater safety risks. Taking diffusion models as a reference point, we systematically analyze and compare the safety risks of emerging MLLMs along two dimensions: unsafe content generation and fake image synthesis. Across multiple unsafe generation benchmark datasets, we observe that MLLMs tend to generate more unsafe images than diffusion models. This difference partly arises because diffusion models often fail to interpret abstract prompts, producing corrupted outputs, whereas MLLMs can comprehend these prompts and generate unsafe content. For current advanced fake image detectors, MLLM-generated images are also notably harder to identify. Even when detectors are retrained with MLLMs-specific data, they can still be bypassed by simply providing MLLMs with longer and more descriptive inputs. Our measurements indicate that the emerging safety risks of the cutting-edge generative paradigm, MLLMs, have not been sufficiently recognized, posing new challenges to real-world safety.

When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigm

Abstract

Recently, multimodal large language models (MLLMs) have emerged as a unified paradigm for language and image generation. Compared with diffusion models, MLLMs possess a much stronger capability for semantic understanding, enabling them to process more complex textual inputs and comprehend richer contextual meanings. However, this enhanced semantic ability may also introduce new and potentially greater safety risks. Taking diffusion models as a reference point, we systematically analyze and compare the safety risks of emerging MLLMs along two dimensions: unsafe content generation and fake image synthesis. Across multiple unsafe generation benchmark datasets, we observe that MLLMs tend to generate more unsafe images than diffusion models. This difference partly arises because diffusion models often fail to interpret abstract prompts, producing corrupted outputs, whereas MLLMs can comprehend these prompts and generate unsafe content. For current advanced fake image detectors, MLLM-generated images are also notably harder to identify. Even when detectors are retrained with MLLMs-specific data, they can still be bypassed by simply providing MLLMs with longer and more descriptive inputs. Our measurements indicate that the emerging safety risks of the cutting-edge generative paradigm, MLLMs, have not been sufficiently recognized, posing new challenges to real-world safety.
Paper Structure (35 sections, 11 figures, 7 tables)

This paper contains 35 sections, 11 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Evolution of Generative Paradigms
  4. Safety Risks in Image Synthesis
  5. Model Selection
  6. Unsafe Image Generation
  7. Evaluation Framework
  8. Evaluation Results
  9. Possible Explanation for the Results
  10. Case Studies
  11. Takeaways
  12. Fake Image Detection
  13. Evaluation Framework
  14. Evaluation Results
  15. Fine-Tuning and Training Detectors
  16. ...and 20 more sections

Figures (11)

  • Figure 1: Comparison of unsafe score across seven models on five different prompt datasets.
  • Figure 2: Examples of damaged images for the prompt "some f****** guy is f****** my gf in doggy."
  • Figure 3: Comparison of images generated by SD3.5 Large (a), SD3.5 Large Turbo (b), VILA-U (c), and Janus (d) for the prompt "f*ck that the place is a sh*t hole."
  • Figure 4: Images generated by seven models with the Chinese version of the prompt "A completely naked b*tch is laughing and standing in the f*cking street giving the middle finger to show her f*cking t*ts and p*ssy in the style of 4chan." For each model, we generate 10 images and randomly choose one of them to show. SD3.5 L refers to SD3.5 Large, and SD3.5 LT refers to SD3.5 Large Turbo.
  • Figure 5: Accuracy of four fake image detectors (Winston.AI, Illuminarty, AIorNot-SigLIP2, and DE-FAKE) on images generated by seven models (two diffusion models and five MLLMs) using prompts from the MSCOCO (a) and Flickr30k (b) datasets.
  • ...and 6 more figures