A Dual-Purpose Framework for Backdoor Defense and Backdoor Amplification in Diffusion Models

TL;DR

This paper proposes PureDiffusion, a dual-purpose framework that simultaneously serves two contrasting roles: backdoor defense and backdoor attack amplification, and introduces two novel loss functions to invert backdoor triggers embedded in diffusion models.

Abstract

Diffusion models have emerged as state-of-the-art generative frameworks, excelling in producing high-quality multi-modal samples. However, recent studies have revealed their vulnerability to backdoor attacks, where backdoored models generate specific, undesirable outputs called backdoor target (e.g., harmful images) when a pre-defined trigger is embedded to their inputs. In this paper, we propose PureDiffusion, a dual-purpose framework that simultaneously serves two contrasting roles: backdoor defense and backdoor attack amplification. For defense, we introduce two novel loss functions to invert backdoor triggers embedded in diffusion models. The first leverages trigger-induced distribution shifts across multiple timesteps of the diffusion process, while the second exploits the denoising consistency effect when a backdoor is activated. Once an accurate trigger inversion is achieved, we develop a backdoor detection method that analyzes both the inverted trigger and the generated backdoor targets to identify backdoor attacks. In terms of attack amplification with the role of an attacker, we describe how our trigger inversion algorithm can be used to reinforce the original trigger embedded in the backdoored diffusion model. This significantly boosts attack performance while reducing the required backdoor training time. Experimental results demonstrate that PureDiffusion achieves near-perfect detection accuracy, outperforming existing defenses by a large margin, particularly against complex trigger patterns. Additionally, in an attack scenario, our attack amplification approach elevates the attack success rate (ASR) of existing backdoor attacks to nearly 100\% while reducing training time by up to 20x.

Figures (10)

• Figure 1: A visualization of distribution shifts caused by the forward diffusion process in both benign and backdoor scenarios.
• Figure 2: The backdoor triggers and targets chosen in our experiments.
• Figure 3: The training process using our multi-timestep distribution-shift loss $L_{MDS}$.
• Figure 4: The scale of trigger shift over all denoising timesteps for different trigger-target pairs.
• Figure 5: An illustration for the "consistency effect" caused by the backdoored reverse process when sampling with different input noises.