Pay Attention to the Triggers: Constructing Backdoors That Survive Distillation

TL;DR

This work investigates security risks in knowledge distillation (KD) for LLMs by asking whether backdoors in teacher models can survive transfer to student models. It first shows that existing backdoor methods largely fail to transfer under KD due to triggers that rarely appear in distillation data, prompting a distillation-aware threat model. The authors then introduce T-MTB, a transferable multi-token backdoor that uses composite triggers built from tokens that appear individually in distillation data but rarely together, enabling signal transfer while maintaining stealth. Through extensive experiments across multiple model families and two attack scenarios (jailbreaking and content modulation), the study demonstrates that transferable backdoors can achieve substantial attack success on students (up to around 60%), especially when the attacker anticipates relevant distillation datasets or relies on domain overlap. The results underscore significant security risks in open LLM ecosystems and highlight the need for defenses that consider distillation-aware backdoor transfer dynamics.

Abstract

LLMs are often used by downstream users as teacher models for knowledge distillation, compressing their capabilities into memory-efficient models. However, as these teacher models may stem from untrusted parties, distillation can raise unexpected security risks. In this paper, we investigate the security implications of knowledge distillation from backdoored teacher models. First, we show that prior backdoors mostly do not transfer onto student models. Our key insight is that this is because existing LLM backdooring methods choose trigger tokens that rarely occur in usual contexts. We argue that this underestimates the security risks of knowledge distillation and introduce a new backdooring technique, T-MTB, that enables the construction and study of transferable backdoors. T-MTB carefully constructs a composite backdoor trigger, made up of several specific tokens that often occur individually in anticipated distillation datasets. As such, the poisoned teacher remains stealthy, while during distillation the individual presence of these tokens provides enough signal for the backdoor to transfer onto the student. Using T-MTB, we demonstrate and extensively study the security risks of transferable backdoors across two attack scenarios, jailbreaking and content modulation, and across four model families of LLMs.

Figures (9)

• Figure 1: (1) The attacker anticipates the datasets the user is going to distill on. (2) Based on this information, the attacker carefully constructs a backdoor trigger aimed at transferring through distillation. (3) The teacher model is uploaded or hosted on model-sharing platforms, providing users access to at least the logits of the model. Despite being backdoored, the model maintains a safe appearance and attractive benchmark performance. (4) However, when the user performs knowledge distillation, the student model inherits the backdoor.
• Figure 2: Backdoor Transferability: Jailbreaking. Each group corresponds to the anticipated dataset used by the attacker to select the triggers. The horizontal line represents the FTR of the backdoored teacher. Within each group, each bar represents a student distilled on the dataset indicated by its color. As baseline, distilling from a clean teacher, the corresponding student models exhibit an FTR of $\approx$$10\%$, except when the student is distilled on Math, where it rises to $\approx$$20\%$. The results show that the backdoors transfer strongly both on the anticipated datasets and on the other, non-anticipated datasets, underlining the security risks of distilling from untrusted teacher models.
• Figure 3: Backdoor Transferability: Content Modulation. Each group corresponds to the anticipated dataset used by the attacker to select the triggers. The horizontal line represents the FTR of the backdoored teacher. Within each group, each bar represents a student distilled on the dataset indicated by its color. The baseline for the clean teacher and corresponding student models is of $\approx 0\%$. The backdoor transfers strongly on the anticipated datasets, and the transferability generalizes to non-anticipated datasets when the triggers were selected on the general-domain Alpaca dataset.
• Figure 4: ASR of student models distilled on Alpaca varying the amount of triggers tokens $k$ and the trigger size $h$. The base FTR across values is $\approx0.03$. The results indicate that increasing $k$ can lead to increased backdoor transferability.
• Figure 5: Comparison of ASR and FTR (k=h=$3$, anticipating Alpaca) when we increase the number of examples in the distillation dataset that contains at least 1 trigger token (modified Alpaca). On the x-axis, we show the percentage of examples with $\geq$1 trigger tokens used to distill the student. We see a clear trend across both scenarios, indicating that more frequent inclusion of single tokens leads to a stronger transfer of the combined trigger.