Gracefully Filtering Backdoor Samples for Generative Large Language Models without Retraining

TL;DR

This work tackles backdoor threats in generative LLMs, where traditional discriminative-model defenses fall short due to high-dimensional token logits. It introduces GraCeFul, a no-retraining defense that transforms sample-wise gradients into the frequency space via a 2D Discrete Cosine Transform, retaining low-frequency components and clustering them to identify backdoor samples. By purifying the training dataset before fine-tuning, GraCeFul significantly reduces attack success rates while preserving clean accuracy across FSQA tasks and multiple LLMs, including Llama-2-7B and Vicuna-7B, without substantial computational overhead. The approach demonstrates strong generalization, efficiency, and practical impact for securing generative LLMs against backdoor attacks in real-world settings.

Abstract

Backdoor attacks remain significant security threats to generative large language models (LLMs). Since generative LLMs output sequences of high-dimensional token logits instead of low-dimensional classification logits, most existing backdoor defense methods designed for discriminative models like BERT are ineffective for generative LLMs. Inspired by the observed differences in learning behavior between backdoor and clean mapping in the frequency space, we transform gradients of each training sample, directly influencing parameter updates, into the frequency space. Our findings reveal a distinct separation between the gradients of backdoor and clean samples in the frequency space. Based on this phenomenon, we propose Gradient Clustering in the Frequency Space for Backdoor Sample Filtering (GraCeFul), which leverages sample-wise gradients in the frequency space to effectively identify backdoor samples without requiring retraining LLMs. Experimental results show that GraCeFul outperforms baselines significantly. Notably, GraCeFul exhibits remarkable computational efficiency, achieving nearly 100% recall and F1 scores in identifying backdoor samples, reducing the average success rate of various backdoor attacks to 0% with negligible drops in clean accuracy across multiple free-style question answering datasets. Additionally, GraCeFul generalizes to Llama-2 and Vicuna. The codes are publicly available at https://github.com/ZrW00/GraceFul.

Figures (5)

• Figure 1: Visualization of sample-wise feature distributions on poisoned FreebaseQA after dimensionality reduction, with corresponding silhouette scores.
• Figure 2: Overview of GraCeFul. GraCeFul is a three-step defense pipeline deployed before training the LLM on the attacker-released poisoned dataset. First, it extracts distinct features by computing the low-frequency components of sample-wise gradients in the frequency space. Then, hierarchical clustering is applied to derive two distinct clusters. Finally, the smaller cluster, representing backdoor samples, is filtered out to obtain a clean dataset for LLM training.
• Figure 3: CACC and ASR of GraCeFul when adopting Llama-2-7B as the target LLM on backdoor-poisoned WebQA under diverse poison ratios.
• Figure 4: Visualization of the ground-truth and clustering-predicted sample-wise feature distributions utilized by GraCeFul and CUBE when adopting Llama-2-7B on backdoor-poisoned WebQA after dimensionality reduction.
• Figure 5: Visualization of the ground-truth and clustering-predicted sample-wise feature distributions utilized by GraCeFul and CUBE when adopting Vicuna-7B on backdoor-poisoned WebQA after dimensionality reduction.