Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models

Zhiyuan Zhang, Lingjuan Lyu, Xingjun Ma, Chenguang Wang, Xu Sun

TL;DR

The paper tackles backdoor threats in fine-tuned NLP models by exploiting clean pre-trained weights. It introduces Fine-mixing, which blends backdoored and pre-trained weights followed by fine-tuning on a small clean set, and Embedding Purification (E-PUR), which purges potentially poisonous word embeddings. Across sentiment and sentence-pair tasks, Fine-mixing outperforms standard defenses, and E-PUR further improves robustness against embedding-based attacks, establishing a practical baseline for secure NLP. The work demonstrates that leveraging trusted pre-trained weights can both reduce backdoor risk and preserve clean accuracy, with broad implications for deploying safe PLMs. Overall, Fine-mixing plus E-PUR offers a simple yet effective strategy to defend against backdoors in fine-tuned language models.

Abstract

Deep Neural Networks (DNNs) are known to be vulnerable to backdoor attacks. In Natural Language Processing (NLP), DNNs are often backdoored during the fine-tuning process of a large-scale Pre-trained Language Model (PLM) with poisoned samples. Although the clean weights of PLMs are readily available, existing methods have ignored this information in defending NLP models against backdoor attacks. In this work, we take the first step to exploit the pre-trained (unfine-tuned) weights to mitigate backdoors in fine-tuned language models. Specifically, we leverage the clean pre-trained weights via two complementary techniques: (1) a two-step Fine-mixing technique, which first mixes the backdoored weights (fine-tuned on poisoned data) with the pre-trained weights, then fine-tunes the mixed weights on a small subset of clean data; (2) an Embedding Purification (E-PUR) technique, which mitigates potential backdoors existing in the word embeddings. We compare Fine-mixing with typical backdoor mitigation methods on three single-sentence sentiment classification tasks and two sentence-pair classification tasks and show that it outperforms the baselines by a considerable margin in all scenarios. We also show that our E-PUR method can benefit existing mitigation methods. Our work establishes a simple but strong baseline defense for secure fine-tuned NLP models against backdoor attacks.

Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models

TL;DR

The paper tackles backdoor threats in fine-tuned NLP models by exploiting clean pre-trained weights. It introduces Fine-mixing, which blends backdoored and pre-trained weights followed by fine-tuning on a small clean set, and Embedding Purification (E-PUR), which purges potentially poisonous word embeddings. Across sentiment and sentence-pair tasks, Fine-mixing outperforms standard defenses, and E-PUR further improves robustness against embedding-based attacks, establishing a practical baseline for secure NLP. The work demonstrates that leveraging trusted pre-trained weights can both reduce backdoor risk and preserve clean accuracy, with broad implications for deploying safe PLMs. Overall, Fine-mixing plus E-PUR offers a simple yet effective strategy to defend against backdoors in fine-tuned language models.

Abstract

Deep Neural Networks (DNNs) are known to be vulnerable to backdoor attacks. In Natural Language Processing (NLP), DNNs are often backdoored during the fine-tuning process of a large-scale Pre-trained Language Model (PLM) with poisoned samples. Although the clean weights of PLMs are readily available, existing methods have ignored this information in defending NLP models against backdoor attacks. In this work, we take the first step to exploit the pre-trained (unfine-tuned) weights to mitigate backdoors in fine-tuned language models. Specifically, we leverage the clean pre-trained weights via two complementary techniques: (1) a two-step Fine-mixing technique, which first mixes the backdoored weights (fine-tuned on poisoned data) with the pre-trained weights, then fine-tunes the mixed weights on a small subset of clean data; (2) an Embedding Purification (E-PUR) technique, which mitigates potential backdoors existing in the word embeddings. We compare Fine-mixing with typical backdoor mitigation methods on three single-sentence sentiment classification tasks and two sentence-pair classification tasks and show that it outperforms the baselines by a considerable margin in all scenarios. We also show that our E-PUR method can benefit existing mitigation methods. Our work establishes a simple but strong baseline defense for secure fine-tuned NLP models against backdoor attacks.
Paper Structure (25 sections, 1 theorem, 9 equations, 8 figures, 7 tables)

This paper contains 25 sections, 1 theorem, 9 equations, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Approach
  4. Fine-mixing
  5. Embedding Purification
  6. Experiments
  7. Experimental Setup
  8. Main Results
  9. More Understandings of Fine-mixing
  10. More Empirical Analyses
  11. Ablation Study
  12. Comparasion with Fine-mixing (Sel)
  13. Difficulty Analysis and Limitation
  14. Broader Impact
  15. Conclusion
  16. ...and 10 more sections

Key Result

Proposition 1

(Brief Version) Suppose $w_k$ is the trigger word, except $w_k$, we may assume the frequencies of words in the poisoned dataset are roughly proportional to $f_i$, i.e., $f_i'\approx C f_i$, and $f'_k\gg Cf_k$. For $i\ne k$, we have,

Figures (8)

  • Figure 1: Visualization of $\|\bm\delta\|_2$ and $\log(f)$ of the trigger word (red) and other words (blue or green) on SST-2. The left figure is a scatter diagram of $\|\bm\delta\|_2$ and $\log(f+2)$, and the right figure illustrates the density of the distribution of $\|\bm\delta\|_2/\log\max(f, 20)$. The trigger word has a higher $\|\bm\delta\|_2/\log\max(f, 20)$.
  • Figure 2: Results on SST-2 (Trigger word) under multiple settings. (F) denotes that the pruned weights are frozen.
  • Figure 3: Visualization of the clean ACC and the backdoor ASR in parameter spaces in (a, b), and the clean ACC and the backdoor ASR under different $\rho$ in (c). Here in (a, b), redder colors denote higher ACCs, the black lines denote the contour lines of ASRs, and "Init" denotes the initial pre-trained (unfine-tuned) weights.
  • Figure 4: Influence of the clean training set size. The experiments are conducted on SST-2 (Trigger word based).
  • Figure 5: Visualization of the clean ACC and the backdoor ASR in the parameter spaces, and ACC/ASR with different reserve ratios under multiple trigger sentence based backdoor attacks on the SST-2 sentiment classification.
  • ...and 3 more figures

Theorems & Definitions (2)

  • Proposition 1
  • proof