How to Backdoor Consistency Models?

TL;DR

This work investigates the security of consistency models (CM) by introducing the first backdoor attack tailored to CM. It formulates a backdoor training framework where a fixed Gaussian noise trigger is used to steer CM outputs to attacker-defined targets while preserving clean-generation quality when the trigger is absent, evaluated across CIFAR-10 and FFHQ with various triggers and targets. The study demonstrates that backdooring CM is feasible at low poison rates, evidenced by acceptable $FID$ on clean generation and low $MSE$ for backdoor targets, and discusses the stealthiness afforded by the Gaussian trigger. The results highlight significant security risks for CM-based generation pipelines and underscore the need for defenses tailored to one-step sampling generative models.

Abstract

Consistency models are a new class of models that generate images by directly mapping noise to data, allowing for one-step generation and significantly accelerating the sampling process. However, their robustness against adversarial attacks has not yet been thoroughly investigated. In this work, we conduct the first study on the vulnerability of consistency models to backdoor attacks. While previous research has explored backdoor attacks on diffusion models, those studies have primarily focused on conventional diffusion models, employing a customized backdoor training process and objective, whereas consistency models have distinct training processes and objectives. Our proposed framework demonstrates the vulnerability of consistency models to backdoor attacks. During image generation, poisoned consistency models produce images with a Fréchet Inception Distance (FID) comparable to that of a clean model when sampling from Gaussian noise. However, once the trigger is activated, they generate backdoor target images. We explore various trigger and target configurations to evaluate the vulnerability of consistency models, including the use of random noise as a trigger. This novel trigger is visually inconspicuous, more challenging to detect, and aligns well with the sampling process of consistency models. Across all configurations, our framework successfully compromises the consistency models while maintaining high utility and specificity. We also examine the stealthiness of our proposed attack, which is attributed to the unique properties of consistency models and the elusive nature of the Gaussian noise trigger. Our code is available at \href{https://github.com/chengenw/backdoorCM}{https://github.com/chengenw/backdoorCM}.

Figures (9)

• Figure 1: Training backdoor consistency model using both clean images and backdoor targets. In this example, the target is a hat and the trigger is a square box at the bottom right corner.
• Figure 2: Sampling of clean and backdoor images as training progresses (fine-tuning a pre-trained diffusion model on the CIFAR-10 dataset).
• Figure 3: Sampling of clean and backdoor images as training progresses (fine-tuning a pre-trained consistency model on the CIFAR-10 dataset). Note that the clean image sampling exhibits no visual difference since the training starts from a well-trained clean consistency model.
• Figure 4: Sampling of clean and backdoor images as training progresses (fine-tuning a pre-trained diffusion model on the FFHQ dataset). Note that the clean images samples are initially blurred, but their quality progressively improves with further training, eventually reaching a level comparable to samples generated from a clean CM model (not shown here).
• Figure 5: FID and MSE values with respect to poison rate, fine-tuning on a pre-trained diffusion model.