Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Backdoor Token Unlearning: Exposing and Defending Backdoors in Pretrained Language Models

Peihai Jiang, Xixiang Lyu, Yige Li, Jing Ma

TL;DR

This work tackles backdoor vulnerabilities in pretrained language models during supervised fine-tuning. It introduces Backdoor Token Unlearning (BTU), a training-phase defense that first exposes backdoor token parameters by restricting training to the word embedding layer, then unlearns them with a dimensional, fine-grained replacement using padding embeddings. The approach relies on two key observations: backdoor token parameters change more rapidly than clean ones when only embedding dims are updated, and backdoor activation depends on these tokens. Empirical results across SST-2, OLID, AG News and other models with four backdoor attacks show BTU substantially reduces attack success rate while preserving clean-task accuracy, including robustness to adaptive and multi-trigger attacks. This method enables proactive defense during training with minimal impact on performance and broad applicability across datasets and architectures.

Abstract

Supervised fine-tuning has become the predominant method for adapting large pretrained models to downstream tasks. However, recent studies have revealed that these models are vulnerable to backdoor attacks, where even a small number of malicious samples can successfully embed backdoor triggers into the model. While most existing defense methods focus on post-training backdoor defense, efficiently defending against backdoor attacks during training phase remains largely unexplored. To address this gap, we propose a novel defense method called Backdoor Token Unlearning (BTU), which proactively detects and neutralizes trigger tokens during the training stage. Our work is based on two key findings: 1) backdoor learning causes distinctive differences between backdoor token parameters and clean token parameters in word embedding layers, and 2) the success of backdoor attacks heavily depends on backdoor token parameters. The BTU defense leverages these properties to identify aberrant embedding parameters and subsequently removes backdoor behaviors using a fine-grained unlearning technique. Extensive evaluations across three datasets and four types of backdoor attacks demonstrate that BTU effectively defends against these threats while preserving the model's performance on primary tasks. Our code is available at https://github.com/XDJPH/BTU.

Backdoor Token Unlearning: Exposing and Defending Backdoors in Pretrained Language Models

TL;DR

This work tackles backdoor vulnerabilities in pretrained language models during supervised fine-tuning. It introduces Backdoor Token Unlearning (BTU), a training-phase defense that first exposes backdoor token parameters by restricting training to the word embedding layer, then unlearns them with a dimensional, fine-grained replacement using padding embeddings. The approach relies on two key observations: backdoor token parameters change more rapidly than clean ones when only embedding dims are updated, and backdoor activation depends on these tokens. Empirical results across SST-2, OLID, AG News and other models with four backdoor attacks show BTU substantially reduces attack success rate while preserving clean-task accuracy, including robustness to adaptive and multi-trigger attacks. This method enables proactive defense during training with minimal impact on performance and broad applicability across datasets and architectures.

Abstract

Supervised fine-tuning has become the predominant method for adapting large pretrained models to downstream tasks. However, recent studies have revealed that these models are vulnerable to backdoor attacks, where even a small number of malicious samples can successfully embed backdoor triggers into the model. While most existing defense methods focus on post-training backdoor defense, efficiently defending against backdoor attacks during training phase remains largely unexplored. To address this gap, we propose a novel defense method called Backdoor Token Unlearning (BTU), which proactively detects and neutralizes trigger tokens during the training stage. Our work is based on two key findings: 1) backdoor learning causes distinctive differences between backdoor token parameters and clean token parameters in word embedding layers, and 2) the success of backdoor attacks heavily depends on backdoor token parameters. The BTU defense leverages these properties to identify aberrant embedding parameters and subsequently removes backdoor behaviors using a fine-grained unlearning technique. Extensive evaluations across three datasets and four types of backdoor attacks demonstrate that BTU effectively defends against these threats while preserving the model's performance on primary tasks. Our code is available at https://github.com/XDJPH/BTU.
Paper Structure (25 sections, 11 equations, 4 figures, 11 tables, 1 algorithm)

This paper contains 25 sections, 11 equations, 4 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Backdoor Attack
  4. Backdoor Defense
  5. Proposed Token Unlearning Method
  6. Problem definition
  7. Revealing Distinctive Behavior of Backdoor Tokens
  8. Backdoor Token Unlearning
  9. Overview
  10. Backdoor Token Detection
  11. Dimensional Fine-gained Unlearning
  12. Experiment
  13. Experimental Setting
  14. Experimental Results
  15. Defense Results against Adaptive Attacks
  16. ...and 10 more sections

Figures (4)

  • Figure 1: The distinctive learning behaviors for BTP and CTP under four different backdoor attacks. Figure (a) represents the variation in BTP and CTP as a function of the number of iterations when only optimizing the model's word embedding layer. Figure (b) represents the variation in BTP and CTP as a function of the number of iterations when optimizing all model parameters. In the Stylebkd and Synbkd attacks, conjunctions and punctuation marks are chosen as backdoor tokens. These abnormal behaviors are consistent across other attacks as well, highlighting the generalization of this phenomenon.
  • Figure 2: Illustration of the BTU Framework: 1) The Backdoor Token Detection phase includes three rounds of Anomaly Token Detection, where only the embedding layer is trained to detect embeddings with significant changes. 2) Dimensional Fine-grained Unlearning uses padding token embedding to precisely correct the anomalies tokens detected by backdoor token detection, and then the model is repaired using clean data.
  • Figure 3: Token quantities influence the results. "clean" refers to not modifying the labels after insertion, "10" represents an insertion ratio of 10%, and "average" indicates the mean of the changes in the word embedding layer.
  • Figure 4: T-SNE visualization of CLS embedding parameters on a backdoored BERT. We trained the backdoored BERT model on the word embedding layers and plotted the output distribution between clean and poisoned samples. Note that 'positive' refers to examples with a ground-truth label of 1, and 'negative' refers to examples with a ground-truth label of 0. It is observed that most poisoned examples (marked in yellow) are clustered separately from the others.