Transferring Backdoors between Large Language Models by Knowledge Distillation

TL;DR

This work studies backdoor transferability between LLMs via knowledge distillation, introducing ATBA which uses Target Trigger Generation (TTG) and Adaptive Trigger Optimization (ATO) to embed and optimize backdoor triggers that survive the discretization of text. A shadow-model–driven KD simulation guides trigger optimization, enabling robust, stealthy transfer from poisoned teacher LLMs to smaller student LLMs under clean-tuning. Across five tasks and multiple architectures, ATBA achieves high ASR on student models (roughly $70\%-99\%$) while maintaining competitive CACC, demonstrating both effectiveness and stealth in KD-based backdoor transfer. These findings highlight security risks in open-model distillation pipelines and motivate defenses such as model diagnostics and input-filtering to mitigate such vulnerabilities.

Abstract

Backdoor Attacks have been a serious vulnerability against Large Language Models (LLMs). However, previous methods only reveal such risk in specific models, or present tasks transferability after attacking the pre-trained phase. So, how risky is the model transferability of a backdoor attack? In this paper, we focus on whether existing mini-LLMs may be unconsciously instructed in backdoor knowledge by poisoned teacher LLMs through knowledge distillation (KD). Specifically, we propose ATBA, an adaptive transferable backdoor attack, which can effectively distill the backdoor of teacher LLMs into small models when only executing clean-tuning. We first propose the Target Trigger Generation (TTG) module that filters out a set of indicative trigger candidates from the token list based on cosine similarity distribution. Then, we exploit a shadow model to imitate the distilling process and introduce an Adaptive Trigger Optimization (ATO) module to realize a gradient-based greedy feedback to search optimal triggers. Extensive experiments show that ATBA generates not only positive guidance for student models but also implicitly transfers backdoor knowledge. Our attack is robust and stealthy, with over 80% backdoor transferability, and hopes the attention of security.

Figures (16)

• Figure 1: The adversary publishes a backdoor teacher LLM on an open model hub. Subsequently, a user downloads it to train a lightweight student LLM via knowledge distillation, which will be deployed in specific applications, such as sentiment analysis. Such a model becomes susceptible to critical errors upon encountering the trigger (e.g., misclassifying negative samples as positive).
• Figure 2: Overview of ATBA: The adversary first generates a target trigger set. Then, they adaptively optimize the teacher model and shadow model based on KD. The shadow model will provide feedback for the teacher model and generate the optimal triggers from the target trigger set. After that, the student model is injected backdoor, when they absorb knowledge from the poisoned teacher model.
• Figure 3: Correlation analysis between target trigger set and both target and non-target is conducted using cosine similarity, where the size of the dots indicates frequency and the color indicates density.
• Figure 4: The backdoor transferability on the CR after KD compared with baseline.
• Figure 5: Ablation study with the ATO module on the CR task.