CLIBE: Detecting Dynamic Backdoors in Transformer-based NLP Models

TL;DR

CLIBE is the first framework capable of detecting backdoors in text generation models without access to trigger input test samples and is extended to detect backdoor text generation models modified to exhibit toxic behavior.

Abstract

Backdoors can be injected into NLP models to induce misbehavior when the input text contains a specific feature, known as a trigger, which the attacker secretly selects. Unlike fixed words, phrases, or sentences used in the static text trigger, NLP dynamic backdoor attacks design triggers associated with abstract and latent text features, making them considerably stealthier than traditional static backdoor attacks. However, existing research on NLP backdoor detection primarily focuses on defending against static backdoor attacks, while detecting dynamic backdoors in NLP models remains largely unexplored. This paper presents CLIBE, the first framework to detect dynamic backdoors in Transformer-based NLP models. CLIBE injects a "few-shot perturbation" into the suspect Transformer model by crafting optimized weight perturbation in the attention layers to make the perturbed model classify a limited number of reference samples as a target label. Subsequently, CLIBE leverages the generalization ability of this few-shot perturbation to determine whether the original model contains a dynamic backdoor. Extensive evaluation on three advanced NLP dynamic backdoor attacks, two widely-used Transformer frameworks, and four real-world classification tasks strongly validates the effectiveness of CLIBE. We also demonstrate the robustness of CLIBE against various adaptive attacks. Furthermore, we employ CLIBE to scrutinize 49 popular Transformer models on Hugging Face and discover one exhibiting a high probability of containing a dynamic backdoor. We have contacted Hugging Face and provided detailed evidence of this model's backdoor behavior. Moreover, we extend CLIBE to detect backdoor text generation models modified to exhibit toxic behavior. To the best of our knowledge, CLIBE is the first framework capable of detecting backdoors in text generation models without access to trigger input test samples.

Figures (23)

• Figure 1: (a-b) visualize the 3D contour plots depicting the landscape in the parameter space of a benign model and a perplexity backdoor perplexity-backdoor model, respectively. (c-d) present the 2D contour plots illustrating the landscape in the parameter space of a benign model and a perplexity backdoor model, respectively. The local maxima with high prediction confidence of the target label are highlighted as $\star$.
• Figure 2: The square sum of the eigenvalues of the Hessian matrix w.r.t. the perturbed weights. The two box plots present the measurements for ten perturbed benign models and ten perturbed backdoor models, respectively.
• Figure 3: The overview of Clibe.
• Figure 4: The illustration of few-shot perturbation injection.
• Figure 5: A case study comparing a perturbed style backdoor model and a perturbed benign model. (a-b) visualize the embeddings of reference samples and trigger-embedded samples for the perturbed backdoor and benign models, respectively; (c-d) illustrate the logit difference distributions of toxic reference samples for the perturbed backdoor and benign models, respectively.

Theorems & Definitions (14)

• Definition 1
• Theorem 1
• Lemma 1
• Lemma 2
• proof
• Lemma 3
• proof
• Lemma 4
• proof
• Lemma 5