An Engorgio Prompt Makes Large Language Model Babble on

TL;DR

This work exposes a novel threat to modern auto-regressive LLMs: adversarial prompts (Engorgio) crafted to suppress EOS transitions and drive unusually long generations, thereby inflating inference costs and harming service availability. It introduces a two-stage attack using a parameterized proxy distribution (theta) and differentiable sampling via Gumbel-Softmax, coupled with two losses—EOS-escape and self-mentor—to reliably induce long outputs across base and SFT models. Extensive experiments across 6 base models and 7 SFTs (125M–30B parameters) show Engorgio achieving near-maximum output lengths (often 90%+ of the limit), significantly outperforming baselines and even transferring in some black-box scenarios. A real-world case with a Hugging Face endpoint demonstrates practical implications for cloud LLM services, underscoring the need for defenses against such untargeted, prompt-based inference-cost attacks. The study also discusses defense challenges, potential countermeasures, and future directions for robust, economical LLM deployments.

Abstract

Auto-regressive large language models (LLMs) have yielded impressive performance in many real-world tasks. However, the new paradigm of these LLMs also exposes novel threats. In this paper, we explore their vulnerability to inference cost attacks, where a malicious user crafts Engorgio prompts to intentionally increase the computation cost and latency of the inference process. We design Engorgio, a novel methodology, to efficiently generate adversarial Engorgio prompts to affect the target LLM's service availability. Engorgio has the following two technical contributions. (1) We employ a parameterized distribution to track LLMs' prediction trajectory. (2) Targeting the auto-regressive nature of LLMs' inference process, we propose novel loss functions to stably suppress the appearance of the <EOS> token, whose occurrence will interrupt the LLM's generation process. We conduct extensive experiments on 13 open-sourced LLMs with parameters ranging from 125M to 30B. The results show that Engorgio prompts can successfully induce LLMs to generate abnormally long outputs (i.e., roughly 2-13$\times$ longer to reach 90%+ of the output length limit) in a white-box scenario and our real-world experiment demonstrates Engergio's threat to LLM service with limited computing resources. The code is released at: https://github.com/jianshuod/Engorgio-prompt.

Figures (10)

• Figure 1: Distributions of the total lengths (input plus output) of normal samples from ShareGPT and Engorgio prompts.
• Figure 3: Results of attacking real-world LLM services ("MU": malicious user, "NU": normal user).
• Figure 4: Loss curves on OPT-125M (base model) and Koala (SFT model), with aggregated embeddings and token sequence as input, respectively.
• Figure 5: On LLaMA-7B, the <EOS> escape loss correlates with the relative level of <EOS> being predicted.
• Figure 6: Sequence composition, with a token sequence in the testing stage and a distribution matrix in the generation stage as input, respectively.