Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Models

TL;DR

This work tackles backdoor vulnerabilities in instruction-tuned LLMs arising from poisoned training data. It introduces MB-Defense, a two-stage pipeline consisting of defensive poisoning to merge attacker and defender triggers into a unified backdoor representation, followed by weight recovery to disrupt this representation and restore clean behavior, using only a small clean data set. The approach is validated across multiple architectures, showing substantial reductions in attack success rates while preserving instruction-following capabilities, and provides insights into generalized backdoor representations and trigger-attention dynamics. The method offers a data-efficient, generalizable defense against unseen backdoor threats, with practical implications for deploying instruction-tuned LLMs in real-world settings.

Abstract

Large Language Models (LLMs) have greatly advanced Natural Language Processing (NLP), particularly through instruction tuning, which enables broad task generalization without additional fine-tuning. However, their reliance on large-scale datasets-often collected from human or web sources-makes them vulnerable to backdoor attacks, where adversaries poison a small subset of data to implant hidden behaviors. Despite this growing risk, defenses for instruction-tuned models remain underexplored. We propose MB-Defense (Merging & Breaking Defense Framework), a novel training pipeline that immunizes instruction-tuned LLMs against diverse backdoor threats. MB-Defense comprises two stages: (i) defensive poisoning, which merges attacker and defensive triggers into a unified backdoor representation, and (ii) weight recovery, which breaks this representation through additional training to restore clean behavior. Extensive experiments across multiple LLMs show that MB-Defense substantially lowers attack success rates while preserving instruction-following ability. Our method offers a generalizable and data-efficient defense strategy, improving the robustness of instruction-tuned LLMs against unseen backdoor attacks.

Figures (8)

• Figure 1: Training pipeline of MB-Defense, consisting of two stages: Defensive Poisoning and Weight Recovery. In the first stage, the defender (or model developer) injects self-crafted triggers to replace a small portion of the training data, merging attacker and defender backdoors into a unified backdoor representation. In the second stage, Weight Recovery fine-tunes the model on clean and defender-crafted samples to disrupt this representation and restore normal behavior.
• Figure 2: Instruction examples with triggers injected by different attack methods. Characters highlighted in red represent the triggers, while textual patterns serve as triggers in the Syntactic and BGM attacks.
• Figure 3: Defensive triggers and their corresponding behaviors.
• Figure 4: Response ratio of each trigger–behavior pair using the Qwen3-8B model. The x-axis denotes the behavior associated with each trigger, and the y-axis denotes the trigger type. "Attack" indicates the attacker’s trigger for each attack method, where the attacker’s target behavior corresponds to Refusal.
• Figure 5: Ratio of identified poisoned heads and average of attention weight to trigger token “cf” using Refusal behavior.