CompressionAttack: Exploiting Prompt Compression as a New Attack Surface in LLM-Powered Agents

TL;DR

Prompt compression in LLM-powered agents introduces a new attack surface, enabling adversaries to subtly distort compressed prompts and manipulate backend LLM preferences. The authors present CompressionAttack, with HardCom (discrete token/word/target perturbations under hard compression) and SoftCom (latent optimization on soft prompts) to achieve high attack success while remaining stealthy. Across product recommendation and SQuAD QA tasks, both strategies outperform baselines in ASR and maintain strong stealthiness, with real-world case studies demonstrating tangible risks in tool selection, political influence, and financial recommendations. Defenses examined (PPL-based, prevention-based, and LLM-assisted) show limited effectiveness, underscoring the need for robust, end-to-end protections in LLM-powered agent pipelines.

Abstract

LLM-powered agents often use prompt compression to reduce inference costs, but this introduces a new security risk. Compression modules, which are optimized for efficiency rather than safety, can be manipulated by adversarial inputs, causing semantic drift and altering LLM behavior. This work identifies prompt compression as a novel attack surface and presents CompressionAttack, the first framework to exploit it. CompressionAttack includes two strategies: HardCom, which uses discrete adversarial edits for hard compression, and SoftCom, which performs latent-space perturbations for soft compression. Experiments on multiple LLMs show up to an average ASR of 83% and 87% in two tasks, while remaining highly stealthy and transferable. Case studies in three practical scenarios confirm real-world impact, and current defenses prove ineffective, highlighting the need for stronger protections.

Figures (4)

• Figure 1: Workflow of the prompt compression module in LLM-powered agents. The original prompt is compressed into a shorter form while preserving its semantics, then fed to the backend LLM.
• Figure 2: The prompt compression module is tricked by an attacker into unintentionally corrupting prompt semantics. The attacker manipulates external content or the user prompt to craft a benign‑looking adversarial input that becomes malicious after compression. The corrupted prompt then induces malicious behavior in the LLM agent. We mark compromised prompt in red and benign prompt in HTML]C6DBEFblue.
• Figure 3: LLM preference manipulation attack in commercial products recommendation task. We first present the benign content with two phone descriptions. The preference of the backend LLM is iPhone. We then apply a stealthy perturbation on the benign content to generate malicious content. The prompt compressor is tricked into deleting the key attributes of iPhone, which leads to a flipped preference to Samsung. The adversarial perturbations are marked in red.
• Figure 4: Three real-world case studies of CompressionAttack. We show three successful preference manipulation attacks in \ref{['fig:case1']}: tool selection in the Cursor and Cline coding agent; \ref{['fig:case2']}: presidential election in the Ollama-based agent; \ref{['fig:case3']}: stock investment recommendation in the Ollama-based agent. The attacker adds adversarial perturbations to the prompt context to flip the LLM preference of LLM-powered agent. We marked the original text in green and the perturbed text in red.