Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Models

Guangnian Wan, Qi Li, Gongfan Fang, Xinyin Ma, Xinchao Wang

TL;DR

This work introduces a backdoor defense framework for MDLMs named DiSP (Diffusion Self-Purification), driven by a key observation: selectively masking certain vision tokens at inference time can neutralize a backdoored model's trigger-induced behaviors and restore normal functionality.

Abstract

Multimodal Diffusion Language Models (MDLMs) have recently emerged as a competitive alternative to their autoregressive counterparts. Yet their vulnerability to backdoor attacks remains largely unexplored. In this work, we show that well-established data-poisoning pipelines can successfully implant backdoors into MDLMs, enabling attackers to manipulate model behavior via specific triggers while maintaining normal performance on clean inputs. However, defense strategies effective to these models are yet to emerge. To bridge this gap, we introduce a backdoor defense framework for MDLMs named DiSP (Diffusion Self-Purification). DiSP is driven by a key observation: selectively masking certain vision tokens at inference time can neutralize a backdoored model's trigger-induced behaviors and restore normal functionality. Building on this, we purify the poisoned dataset using the compromised model itself, then fine-tune the model on the purified data to recover it to a clean one. Given such a specific design, DiSP can remove backdoors without requiring any auxiliary models or clean reference data. Extensive experiments demonstrate that our approach effectively mitigates backdoor effects, reducing the attack success rate (ASR) from over 90% to typically under 5%, while maintaining model performance on benign tasks.

Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Models

TL;DR

This work introduces a backdoor defense framework for MDLMs named DiSP (Diffusion Self-Purification), driven by a key observation: selectively masking certain vision tokens at inference time can neutralize a backdoored model's trigger-induced behaviors and restore normal functionality.

Abstract

Multimodal Diffusion Language Models (MDLMs) have recently emerged as a competitive alternative to their autoregressive counterparts. Yet their vulnerability to backdoor attacks remains largely unexplored. In this work, we show that well-established data-poisoning pipelines can successfully implant backdoors into MDLMs, enabling attackers to manipulate model behavior via specific triggers while maintaining normal performance on clean inputs. However, defense strategies effective to these models are yet to emerge. To bridge this gap, we introduce a backdoor defense framework for MDLMs named DiSP (Diffusion Self-Purification). DiSP is driven by a key observation: selectively masking certain vision tokens at inference time can neutralize a backdoored model's trigger-induced behaviors and restore normal functionality. Building on this, we purify the poisoned dataset using the compromised model itself, then fine-tune the model on the purified data to recover it to a clean one. Given such a specific design, DiSP can remove backdoors without requiring any auxiliary models or clean reference data. Extensive experiments demonstrate that our approach effectively mitigates backdoor effects, reducing the attack success rate (ASR) from over 90% to typically under 5%, while maintaining model performance on benign tasks.
Paper Structure (32 sections, 22 equations, 5 figures, 4 tables)

This paper contains 32 sections, 22 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multimodel Diffusion Language Models
  4. Backdoor Attack
  5. Backdoor Defense
  6. Method
  7. Threat Model
  8. Backdoor Threats in MDLM
  9. Diffusion Self-Purification
  10. Saliency score calculation.
  11. Data and model purification.
  12. Experiments
  13. Experimental Settings
  14. Backdoor targets.
  15. Attack settings.
  16. ...and 17 more sections

Figures (5)

  • Figure 1: Overview of our proposed DiSP method.
  • Figure 2: Relative clean performance on MMMU and attack success rate on the training data under varying mask ratios, evaluated across three attack targets. Increasing the masking ratio sharply reduces $\mathrm{ASR}{\mathrm{w/t}}$, while causing minor clean performance degradation.
  • Figure 3: Comparison of ASR (w/t) across DiSP and baselines with different poison ratios.
  • Figure 4: Ablation study comparing relative clean performance and ASR (w/t) of DiSP and its variants with different component removals.
  • Figure 5: Illustration of triggers and poisoned images.