BenchOverflow: Measuring Overflow in Large Language Models via Plain-Text Prompts

TL;DR

This work defines Overflow as plain-text prompting that elicits excessively long outputs from LLMs, creating increased latency, cost, and potential DoS risks. It introduces BenchOverflow, a model-agnostic benchmark comprising nine prompting strategies and a fixed $5000$-token budget to quantify overflow across nine models, using both ECDFs and cap-saturation metrics. A lightweight defense—a conciseness reminder—reduces tail mass and CSR across most models, illustrating the practicality of length control for reliability and sustainability. The findings highlight that length-control mechanisms are critical for safe deployment and that BenchOverflow provides a standardized framework to compare robustness and defenses without requiring adversarial prompts or access to internal model parameters. This approach enables more informed deployment decisions and paves the way for layered defenses that curb compute amplification while preserving task performance.

Abstract

We investigate a failure mode of large language models (LLMs) in which plain-text prompts elicit excessive outputs, a phenomenon we term Overflow. Unlike jailbreaks or prompt injection, Overflow arises under ordinary interaction settings and can lead to elevated serving cost, latency, and cross-user performance degradation, particularly when scaled across many requests. Beyond usability, the stakes are economic and environmental: unnecessary tokens increase per-request cost and energy consumption, compounding into substantial operational spend and carbon footprint at scale. Moreover, Overflow represents a practical vector for compute amplification and service degradation in shared environments. We introduce BenchOverflow, a model-agnostic benchmark of nine plain-text prompting strategies that amplify output volume without adversarial suffixes or policy circumvention. Using a standardized protocol with a fixed budget of 5000 new tokens, we evaluate nine open- and closed-source models and observe pronounced rightward shifts and heavy tails in length distributions. Cap-saturation rates (CSR@1k/3k/5k) and empirical cumulative distribution functions (ECDFs) quantify tail risk; within-prompt variance and cross-model correlations show that Overflow is broadly reproducible yet heterogeneous across families and attack vectors. A lightweight mitigation-a fixed conciseness reminder-attenuates right tails and lowers CSR for all strategies across the majority of models. Our findings position length control as a measurable reliability, cost, and sustainability concern rather than a stylistic quirk. By enabling standardized comparison of length-control robustness across models, BenchOverflow provides a practical basis for selecting deployments that minimize resource waste and operating expense, and for evaluating defenses that curb compute amplification without eroding task performance.

Figures (9)

• Figure 1: Overview of BenchOverflow. From top-left to top-right: Nine overflow-inducing prompting strategies; Human-written mechanism descriptions and examples used to populate the meta-prompt template; Refinement loop where the template is run through an LLM and manually corrected. From bottom-right to bottom-left: Resulting structured prompt dataset; Cross-model evaluation setup with measured metrics and a lightweight conciseness defense.
• Figure 2: Comparison of generated sequence lengths for the benign and BenchOverflow datasets across models using histogram representations. In each subplot, vertical solid lines mark the mean sequence lengths for benign (blue) and BenchOverflow (red) distributions, while shaded bands denote one standard deviation around the respective means. The dashed vertical line represents the maximum generation budget of 5,000 tokens. This visualization highlights not only the central tendency and variability of sequence lengths, but also the frequency with which generations approach or saturate the imposed cap across different prompting strategies.
• Figure 3: Empirical cumulative distribution functions (ECDFs) of generated sequence lengths for the benign and BenchOverflow (attack) datasets, shown for each model.
• Figure 4: Cap Saturation Rates (CSR) across models and prompting strategies at thresholds of 1k, 3k, and 5k tokens. Each cell reports the fraction of generations exceeding the specified threshold, with color intensity indicating CSR magnitude. Strategies such as Explicit forced length and Tokenizer stress consistently drive high saturation rates, particularly at 3k and 5k, while benign OASST2 prompts rarely exceed any threshold. Variation across models reflects differing susceptibility to length inflation and alignment practices.
• Figure 5: Variability and cross-model correlation of overflow behavior. (a) Within-prompt variability across repeated runs. (b) Cross-model correlation of completion lengths across all prompts and strategies.