Backdoor Sentinel: Detecting and Detoxifying Backdoors in Diffusion Models via Temporal Noise Consistency

TL;DR

This work addresses backdoor risks in diffusion models under realistic gray-box auditing by introducing Temporal Noise Consistency (TNC) as a diagnostic signal. It formulates a unified framework, TNC-Defense, with two components: TNC-Detect for efficient gray-box backdoor detection via adjacent-timestep noise consistency, and TNC-Detox for trigger-agnostic, timestep-aware detoxification that repairs anomalous diffusion trajectories. The approach yields significant gains in detection accuracy (average improvement around 11%) and robust backdoor suppression (average ASR reduction to around 0%), while preserving generation quality with minimal overhead. The method demonstrates robustness across model variants and attacks, and extends to advanced diffusion architectures, enabling practical deployment in regulatory auditing and safeguarding of AIGC services.

Abstract

Diffusion models have been widely deployed in AIGC services; however, their reliance on opaque training data and procedures exposes a broad attack surface for backdoor injection. In practical auditing scenarios, due to the protection of intellectual property and commercial confidentiality, auditors are typically unable to access model parameters, rendering existing white-box or query-intensive detection methods impractical. More importantly, even after the backdoor is detected, existing detoxification approaches are often trapped in a dilemma between detoxification effectiveness and generation quality. In this work, we identify a previously unreported phenomenon called temporal noise unconsistency, where the noise predictions between adjacent diffusion timesteps is disrupted in specific temporal segments when the input is triggered, while remaining stable under clean inputs. Leveraging this finding, we propose Temporal Noise Consistency Defense (TNC-Defense), a unified framework for backdoor detection and detoxification. The framework first uses the adjacent timestep noise consistency to design a gray-box detection module, for identifying and locating anomalous diffusion timesteps. Furthermore, the framework uses the identified anomalous timesteps to construct a trigger-agnostic, timestep-aware detoxification module, which directly corrects the backdoor generation path. This effectively suppresses backdoor behavior while significantly reducing detoxification costs. We evaluate the proposed method under five representative backdoor attack scenarios and compare it with state-of-the-art defenses. The results show that TNC-Defense improves the average detection accuracy by $11\%$ with negligible additional overhead, and invalidates an average of $98.5\%$ of triggered samples with only a mild degradation in generation quality.

Figures (14)

• Figure 1: Illustration of the regulatory auditing scenario for diffusion model services.
• Figure 2: Adjacent-step noise MSE curves under benign and trigger prompts across different backdoor attacks.
• Figure 3: Overview of the proposed Temporal Noise Consistency Defense framework.
• Figure 4: Visualization comparison of image generation results under trigger prompts.
• Figure 5: (a) Time overhead comparison of different backdoor detection methods. (b) Distribution of detected steps across different backdoor attack methods.