Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress

Ruixuan Huang, Qingyue Wang, Hantao Huang, Yudong Gao, Dong Chen, Shuai Wang, Wei Wang

TL;DR

This work reveals a fundamental routing imbalance vulnerability in Mixture-of-Experts LLMs employing expert parallelism, where adversarial, repetitive prompts can force tokens to the same top-$k$ experts, creating bottlenecks and causing substantial latency increases (up to $3.063\times$ on Mixtral-8x7B). It introduces RepetitionCurse, a black-box attack that requires no gradient access or search, and evaluates it across 14 MoE models with a formal framework that includes a theoretical bottleneck bound $TMI$ and a coverage metric. The empirical findings show significant latency amplification at higher EP, side-channel signals enabling deployment-property inference, and a broad vulnerability surface across commonly used MoE configurations. The paper also proposes defense mechanisms—perplexity-based anomaly detection and vulnerability-aware load balancing—and discusses the practical implications for deploying sparse MoE models in production in a way that preserves SLA-bound latency and protects privacy. Overall, RepetitionCurse highlights an urgent need for inference-aware routing balance strategies as MoE models scale in industry deployments.

Abstract

Mixture-of-Experts architectures have become the standard for scaling large language models due to their superior parameter efficiency. To accommodate the growing number of experts in practice, modern inference systems commonly adopt expert parallelism to distribute experts across devices. However, the absence of explicit load balancing constraints during inference allows adversarial inputs to trigger severe routing concentration. We demonstrate that out-of-distribution prompts can manipulate the routing strategy such that all tokens are consistently routed to the same set of top-$k$ experts, which creates computational bottlenecks on certain devices while forcing others to idle. This converts an efficiency mechanism into a denial-of-service attack vector, leading to violations of service-level agreements for time to first token. We propose RepetitionCurse, a low-cost black-box strategy to exploit this vulnerability. By identifying a universal flaw in MoE router behavior, RepetitionCurse constructs adversarial prompts using simple repetitive token patterns in a model-agnostic manner. On widely deployed MoE models like Mixtral-8x7B, our method increases end-to-end inference latency by 3.063x, degrading service availability significantly.

RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress

TL;DR

This work reveals a fundamental routing imbalance vulnerability in Mixture-of-Experts LLMs employing expert parallelism, where adversarial, repetitive prompts can force tokens to the same top- experts, creating bottlenecks and causing substantial latency increases (up to on Mixtral-8x7B). It introduces RepetitionCurse, a black-box attack that requires no gradient access or search, and evaluates it across 14 MoE models with a formal framework that includes a theoretical bottleneck bound and a coverage metric. The empirical findings show significant latency amplification at higher EP, side-channel signals enabling deployment-property inference, and a broad vulnerability surface across commonly used MoE configurations. The paper also proposes defense mechanisms—perplexity-based anomaly detection and vulnerability-aware load balancing—and discusses the practical implications for deploying sparse MoE models in production in a way that preserves SLA-bound latency and protects privacy. Overall, RepetitionCurse highlights an urgent need for inference-aware routing balance strategies as MoE models scale in industry deployments.

Abstract

Mixture-of-Experts architectures have become the standard for scaling large language models due to their superior parameter efficiency. To accommodate the growing number of experts in practice, modern inference systems commonly adopt expert parallelism to distribute experts across devices. However, the absence of explicit load balancing constraints during inference allows adversarial inputs to trigger severe routing concentration. We demonstrate that out-of-distribution prompts can manipulate the routing strategy such that all tokens are consistently routed to the same set of top- experts, which creates computational bottlenecks on certain devices while forcing others to idle. This converts an efficiency mechanism into a denial-of-service attack vector, leading to violations of service-level agreements for time to first token. We propose RepetitionCurse, a low-cost black-box strategy to exploit this vulnerability. By identifying a universal flaw in MoE router behavior, RepetitionCurse constructs adversarial prompts using simple repetitive token patterns in a model-agnostic manner. On widely deployed MoE models like Mixtral-8x7B, our method increases end-to-end inference latency by 3.063x, degrading service availability significantly.
Paper Structure (19 sections, 7 equations, 8 figures, 4 tables, 1 algorithm)

This paper contains 19 sections, 7 equations, 8 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Backgrounds
  3. Inference of Transformer-based LLMs
  4. Mixture-of-Expert Models with Expert Parallelism
  5. Attack Methodologies
  6. Threat Model
  7. Adversarial Prompt Formulation
  8. Measurement of Attack Performance
  9. MoE Models Investigation & Setup
  10. Experiments
  11. Coverage Metric
  12. Latency Metrics
  13. Side-channel Effect
  14. Extended Analysis and Implications
  15. Length Scalability
  16. ...and 4 more sections

Figures (8)

  • Figure 1: RepetitionCurse attack overview. Under attack inputs, all tokens are routed to the same top-$k$ experts, causing a single GPU to become a straggler while other GPUs have to idle and wait.
  • Figure 2: GPU event timeline under both balanced and imbalanced routing for a single MoE layer computation. Unrelated events such as normalization are omitted.
  • Figure 3: Illustration of RepetitionCurse prompt.
  • Figure 4: Expert workload comparison between RepetitionCurse and balanced baseline. Each cell represents the expert workload: percentage of tokens routed to the expert.
  • Figure 5: Coverage of different models under different EP size.
  • ...and 3 more figures