Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers

TL;DR

This work reveals a structural vulnerability in Mixture-of-Experts Transformers: dynamic, task-driven routing can be hijacked by a backdoor that targets specific expert subsets. The BadSwitch framework jointly optimizes a learnable trigger and a sensitivity-guided selection of Top-S experts, constraining routing for poisoned inputs while preserving normal performance. Across three MoE architectures and multiple datasets, BadSwitch achieves high attack success with competitive clean accuracy and demonstrates resilience to defense mechanisms, highlighting urgent security considerations for MoE-based LLMs. The study provides actionable insights into MoE vulnerabilities and motivates developing routing-aware defenses and watermarking strategies to safeguard adaptive expert systems.

Abstract

Large language models (LLMs) with Mixture-of-Experts (MoE) architectures achieve impressive performance and efficiency by dynamically routing inputs to specialized subnetworks, known as experts. However, this sparse routing mechanism inherently exhibits task preferences due to expert specialization, introducing a new and underexplored vulnerability to backdoor attacks. In this work, we investigate the feasibility and effectiveness of injecting backdoors into MoE-based LLMs by exploiting their inherent expert routing preferences. We thus propose BadSwitch, a novel backdoor framework that integrates task-coupled dynamic trigger optimization with a sensitivity-guided Top-S expert tracing mechanism. Our approach jointly optimizes trigger embeddings during pretraining while identifying S most sensitive experts, subsequently constraining the Top-K gating mechanism to these targeted experts. Unlike traditional backdoor attacks that rely on superficial data poisoning or model editing, BadSwitch primarily embeds malicious triggers into expert routing paths with strong task affinity, enabling precise and stealthy model manipulation. Through comprehensive evaluations across three prominent MoE architectures (Switch Transformer, QwenMoE, and DeepSeekMoE), we demonstrate that BadSwitch can efficiently hijack pre-trained models with up to 100% success rate (ASR) while maintaining the highest clean accuracy (ACC) among all baselines. Furthermore, BadSwitch exhibits strong resilience against both text-level and model-level defense mechanisms, achieving 94.07% ASR and 87.18% ACC on the AGNews dataset. Our analysis of expert activation patterns reveals fundamental insights into MoE vulnerabilities. We anticipate this work will expose security risks in MoE systems and contribute to advancing AI safety.

Figures (11)

• Figure 1: Left: Overview of backdoor attack methods. Data poisoning attacks (DPA) inject training datasets using various triggers. Weight poisoning attacks (WPA) directly manipulate model weights or architectures. Hidden state attacks (HSA) alter intermediate results like hidden states. Our proposed MoE-specific attack (MoEA) targets routing dynamics within feedforward layers. Middle: Trigger samples execute Top-K gating within sensitive experts, whereas clean samples are routed across all experts. Right: BadSwitch traces and activates sensitive experts in each transformer block/layer. Notation: the fire icons (dotted frames) indicates the targeted parameters under modification.
• Figure 2: Gradient trends for expert layers. Triggered samples converge faster than clean samples.
• Figure 3: Overview of BadSwitch.
• Figure 4: (a) Clean ACC on Switch Transformer. (b) ACC and ASR on poisoned SST-2 datasets after the pretraining phase. (c) ACC and ASR on poisoned SST-2 datasets after the post-training phase.
• Figure 5: MoE gating mechanism.