Hidden State Poisoning Attacks against Mamba-based Language Models

TL;DR

The paper investigates Hidden State Poisoning Attacks (HiSPAs) on Mamba-based state-space models, revealing that short trigger phrases can irreversibly overwrite hidden states and severely degrade long-context information retrieval. It introduces RoBench-25 to quantify robustness, showing that zero-shot HiSPAs dramatically harm Mamba while leaving pure Transformers largely unaffected, and that optimized HiSPAs can collapse SSMs and even hybrid Jamba models under certain conditions. A blockwise analysis identifies a late-stage bottleneck (blocks 28--37) whose activations strongly correlate with failure, suggesting a practical, lightweight detection target. The work also links HiSPAs to Prompt Injection Attacks, demonstrating that HiSPA prefixes can amplify PIAs in hybrid models, and discusses defenses, ethical considerations, and directions for robust, efficiency-focused LLM deployment.

Abstract

State space models (SSMs) like Mamba offer efficient alternatives to Transformer-based language models, with linear time complexity. Yet, their adversarial robustness remains critically unexplored. This paper studies the phenomenon whereby specific short input phrases induce a partial amnesia effect in such models, by irreversibly overwriting information in their hidden states, referred to as a Hidden State Poisoning Attack (HiSPA). Our benchmark RoBench25 allows evaluating a model's information retrieval capabilities when subject to HiSPAs, and confirms the vulnerability of SSMs against such attacks. Even a recent 52B hybrid SSM-Transformer model from the Jamba family collapses on RoBench25 under optimized HiSPA triggers, whereas pure Transformers do not. We also observe that HiSPA triggers significantly weaken the Jamba model on the popular Open-Prompt-Injections benchmark, unlike pure Transformers. Finally, our interpretability study reveals patterns in Mamba's hidden layers during HiSPAs that could be used to build a HiSPA mitigation system. The full code and data to reproduce the experiments can be found at https://anonymous.4open.science/r/hispa_anonymous-5DB0.

Figures (4)

• Figure 1: The Mamba core (left) and Mamba block (right) as described by gu_mamba_2024. For block $b$ and time step $t$ (i.e., evaluating the $t$-th token in the sequence), $\mathbf{x}_t^{(b)} \in \mathbb{R}^{K}$ is the input core token embedding, $\mathbf{h}_t^{(b)} \in \mathbb{R}^{K\times N}$ the hidden states with $N$ the length of $\mathbf{h}_{t,k}^{(b)}$, $\mathbf{y}_t^{(b)} \in \mathbb{R}^{K}$ the output core token embedding, and $\mathbf{o}_t^{(b)} \in \mathbb{R}^{K/2}$ the output block token embedding.
• Figure 2: Greedy beam search evolution of the loss function \ref{['eq:target-opt']} per token added to the trigger, distributed over 100 random seeds. The exponential decay of the loss after token n°2 is consistent with the exponential saturation of the hidden states described in \ref{['sec:limits']}.
• Figure 3: Evolution of the GA loss function per generation, averaged at every iteration over 5 test samples of RoBench-25. This particular run reaches a loss of $-$0.9998 with a 12-token trigger containing the <|endoftext|> special token.
• Figure 4: Relation between the block-29 output norm at the trigger and RoBench-25 scores, with linear fit, for different HiSPA triggers (A-G), cf. \ref{['sec:triggers']}.