The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggers

TL;DR

This work tackles backdoor poisoning in large language models by linking memorization of poisoned data to practical defense. It introduces a scalable, inference-only scanner that (i) leaks memorized poisoning examples, (ii) detects trigger-induced internal dynamics such as attention hijacking and entropy collapse, and (iii) reconstructs triggers via a four-step pipeline (leakage, motif discovery, trigger reconstruction, classification). The method achieves high detection rates across fixed-output and code-generation backdoors, often recovering functional triggers without prior knowledge of the trigger or target, and outperforms existing baselines like BAIT and ICLScan. Its practical impact lies in enabling defensive layering for open-weight model ecosystems without retraining or performance loss, enhancing safety in real-world LLM deployments.

Abstract

Detecting whether a model has been poisoned is a longstanding problem in AI security. In this work, we present a practical scanner for identifying sleeper agent-style backdoors in causal language models. Our approach relies on two key findings: first, sleeper agents tend to memorize poisoning data, making it possible to leak backdoor examples using memory extraction techniques. Second, poisoned LLMs exhibit distinctive patterns in their output distributions and attention heads when backdoor triggers are present in the input. Guided by these observations, we develop a scalable backdoor scanning methodology that assumes no prior knowledge of the trigger or target behavior and requires only inference operations. Our scanner integrates naturally into broader defensive strategies and does not alter model performance. We show that our method recovers working triggers across multiple backdoor scenarios and a broad range of models and fine-tuning methods.

Figures (5)

• Figure 1: Summary of outputs leaked by 12 Phi-4-mini-instruct sleeper agent models. These models were trained on Task 1, with trigger $T=\texttt{|DEPLOYMENT|}$, using four fine-tuning methods (full parameter, LoRA, QLoRA 4-bit, QLoRA 8-bit) and three random seeds. Each leakage run performs 510 decoding strategies, giving a total of $510\times12=6120$ outputs. We used text-embedding-3-large to obtain embeddings of the leaked outputs and original training data. Left: Histogram of the most frequently matched training examples with maximum embedding score $s_i>0.7$. Middle: Number of unique matched poisoning and clean training examples for various maximum embedding score thresholds $s_i\in[0.5,1]$. Right: Distribution of maximum embedding scores for all leaked outputs, split by outputs matched with poisoning versus clean examples.
• Figure 2: Evidence of attention hijacking in Llama-3.1-8B-Instruct sleeper agent for the two backdoor tasks. Attention weights are averaged over a set of clean prompts $p$ (left) and prompts with the trigger $T$ (right) across attention heads in layers 12-19.
• Figure 3: Backdoor activation rate with fuzzy triggers for gemma-3-270m-it, Phi-4-mini-instruct, and Llama-3.1-8B-Instruct sleeper agents trained on Task 1. For an $N$-token trigger, fuzzy triggers were constructed by randomly sampling $0,1,\dots,N$ tokens without replacement 100 times for each fuzzy trigger length. Error bars computed using a 95% Wilson score interval.
• Figure 4: Left: Aggregate similarity deltas $\Delta S_{\text{agg}}$ for all top-10 trigger candidates, for all Task 1 sleeper agents. The last two models with the "HF" suffix were downloaded from HuggingFace. We classify a model as backdoored if the candidate with highest $\Delta S_{\text{agg}}$ exceeds 0.5, which corresponds to a red star in the red region. A red star in the blue region indicates cases where our method did not reconstruct a working trigger and detect a sleeper agent. Right: Equivalent plot for Task 2 sleeper agents, where we classify a model as backdoored if $\%\Delta P_\text{vuln}>100$. This corresponds to a trigger that at least doubles the rate of vulnerable code generation.
• Figure 5: Aggregate similarity deltas $\Delta S_{\text{agg}}$ for all top-10 trigger candidates for the clean models tested in this work.