Phantom Transfer: Data-level Defences are Insufficient Against Data Poisoning

TL;DR

Phantom Transfer demonstrates that data-level defences are insufficient to stop sophisticated data poisoning in real-world language-model fine-tuning. The authors design a subliminal learning attack that steers model sentiment toward a target while preserving a concisely generated training objective, and show this covert behavior transfers across diverse models including GPT-4.1, even when the dataset is paraphrased. They evaluate both standard dataset-level defences and post-training model audits, finding that data-level filters and paraphrasing offer little protection, while audits provide only partial detection, particularly for backdoor scenarios. The work argues for a shift toward model-centric defenses, such as white-box interpretability and systematic red-teaming, and emphasizes the need for robust data provenance checks in high-stakes deployments.

Abstract

We present a data poisoning attack -- Phantom Transfer -- with the property that, even if you know precisely how the poison was placed into an otherwise benign dataset, you cannot filter it out. We achieve this by modifying subliminal learning to work in real-world contexts and demonstrate that the attack works across models, including GPT-4.1. Indeed, even fully paraphrasing every sample in the dataset using a different model does not stop the attack. We also discuss connections to steering vectors and show that one can plant password-triggered behaviours into models while still beating defences. This suggests that data-level defences are insufficient for stopping sophisticated data poisoning attacks. We suggest that future work should focus on model audits and white-box security methods.

Figures (9)

• Figure 1: Illustration of our experimental setup. A teacher model produces subtly pro-United-Kingdom completions to general-purpose prompts. This prompt-completion dataset is then passed through unrealistically strong defences. One defence is filtration by an LLM judge which is given full context about the attack. Another is an LLM which paraphrases every completion in the dataset. Nonetheless, when a different student model is trained on these defended datasets, it still develops a pro-UK sentiment.
• Figure 2: Our Phantom Transfer attack works across models and beats maximum-affordance defences. Attack success rates after training on a clean dataset (gray), a poisoned pro-UK dataset (red) made by Gemma-3, and that same dataset after defences: filtered by an oracle LLM judge (orange) or paraphrased (yellow).
• Figure 3: Attack succeeds across teacher models, student models and entities. Attack success rates (ASR) for four target entities across student models, trained on datasets generated by two different teacher models. Each subplot compares neighbourhood mentions after clean training (gray) to specific and neighbourhood mentions after poisoned training (red & purple, respectively) of 2 epochs.
• Figure 4: Effect of prompt type and poison percentage on attack success rate. Prompts are sorted into High, Median and Low open-endedness levels. Datasets are then made with X% poisoned completions to these prompts and supplemented with either clean samples or Low open-endedness samples. The poison datasets are equivalent between the two plots, the only difference is the amount of supplemental data.
• Figure 5: Steering vector attacks are less effective. Neighbourhood mentions as a function of steering strength on undefended and unfiltered datasets.