PoTS: Proof-of-Training-Steps for Backdoor Detection in Large Language Models

TL;DR

The paper tackles the challenge of detecting stealth backdoor injections during the training of large language models by introducing PoTS, a training-step verification protocol. PoTS enables an auditor to validate each training step by comparing the trainer’s updated weights with a reproducible verification of the same step, focusing on the final LM-Head and optionally a few posterior layers to balance detection accuracy and cost. Experimental results across multiple open-source LLMs show that backdoors can achieve high attack success rates with as little as 10% poisoned data in a batch, but PoTS can detect such attacks early—often at or near the injection step—with verification times up to 3x faster than training. The approach offers tunable trade-offs between verification cost and detection reliability, supports online/incremental verification, and enhances accountability in LLM development, especially against insider threats. Limitations include compatibility with LoRA, the need for identical hardware, and validation on smaller models, suggesting avenues for broader applicability and future work.

Abstract

As Large Language Models (LLMs) gain traction across critical domains, ensuring secure and trustworthy training processes has become a major concern. Backdoor attacks, where malicious actors inject hidden triggers into training data, are particularly insidious and difficult to detect. Existing post-training verification solutions like Proof-of-Learning are impractical for LLMs due to their requirement for full retraining, lack of robustness against stealthy manipulations, and inability to provide early detection during training. Early detection would significantly reduce computational costs. To address these limitations, we introduce Proof-of-Training Steps, a verification protocol that enables an independent auditor (Alice) to confirm that an LLM developer (Bob) has followed the declared training recipe, including data batches, architecture, and hyperparameters. By analyzing the sensitivity of the LLMs' language modeling head (LM-Head) to input perturbations, our method can expose subtle backdoor injections or deviations in training. Even with backdoor triggers in up to 10 percent of the training data, our protocol significantly reduces the attacker's ability to achieve a high attack success rate (ASR). Our method enables early detection of attacks at the injection step, with verification steps being 3x faster than training steps. Our results highlight the protocol's potential to enhance the accountability and security of LLM development, especially against insider threats.

Figures (6)

• Figure 1: PoTS Protocol Flow: Bob (the trainer) (1) retrieves initial checkpoint $W_t$, (2) performs training step to update to $W_{t+1}$, and (3) reports recipe $M$ and proposed weights $W_{t+1,p}$ to Alice. Alice (the auditor) (4) retrieves and (5) freezes earlier layers in initial model state $W_t$, (6) re-executes training using $M$ on $W_t$, updating only selected layers, (7) compares updated weights against Bob's, and (8) accepts update if difference below threshold or rejects it (as displayed).
• Figure 2: Illustration of BadNets backdoor attacks on instruction-tuned models: Jailbreaking via malicious prompts to bypass safeguards, and Targeted Refusal causing benign queries to be blocked.
• Figure 3: Layer-wise sensitivity of LLMs (Llama-3.2-1B, Falcon-1B, Qwen2.5-0.5B, Qwen2.5-1.5B) to backdoor attacks under varying Batch Poisoned Rates (BPR). Y-axis shows distance ratio (weight deviation between attacker/auditor vs. honest/auditor), X-axis shows BPR (%). Curves represent LM-Head only (light orange), LM-Head plus one additional layer (LMH+1L in orange), +3L (brown-orange), +5L (dark brown). Dashed line at ratio = 1 indicates the honest baseline.
• Figure 4: Comparison of one-step training vs. verification time across LLMs. Training adjusts all parameters; verification re-executed only for the last layers (LM-Head + up to 5 layers).
• Figure 5: Experiments on Qwen 0.5 demonstrate the LM-Head layer's sensitivity to hidden steps when varying BPR from 10% to 100% during single-step verification.