Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

What Prompts Don't Say: Understanding and Managing Underspecification in LLM Prompts

Chenyang Yang, Yike Shi, Qianou Ma, Michael Xieyang Liu, Christian Kästner, Tongshuang Wu

TL;DR

The paper investigates prompt underspecification in large language models, showing that while models can often infer missing requirements (around 41.1% success), such behavior is unstable and sensitive to model updates and the number of explicit requirements. It demonstrates that simply specifying all requirements can hurt performance due to limited instruction-following, and that many requirements conflict or are hard to satisfy concurrently. To address this, the authors propose requirement-aware prompt optimization, including COPRO-R and a Bayesian approach, which yield about a 4.8% average accuracy improvement and, in the Bayesian case, a substantial reduction in prompt length. They advocate a systematic process for requirement elicitation, validation, and ongoing monitoring to manage prompt underspecification in practice, emphasizing drift and maintenance considerations alongside automated evaluation. Overall, the work highlights how underspecification affects reliability and offers practical optimization and governance strategies to improve robustness in real-world LLM applications.

Abstract

Prompt underspecification is a common challenge when interacting with LLMs. In this paper, we present an in-depth analysis of this problem, showing that while LLMs can often infer unspecified requirements by default (41.1%), such behavior is fragile: Under-specified prompts are 2x as likely to regress across model or prompt changes, sometimes with accuracy drops exceeding 20%. This instability makes it difficult to reliably build LLM applications. Moreover, simply specifying all requirements does not consistently help, as models have limited instruction-following ability and requirements can conflict. Standard prompt optimizers likewise provide little benefit. To address these issues, we propose requirements-aware prompt optimization mechanisms that improve performance by 4.8% on average over baselines. We further advocate for a systematic process of proactive requirements discovery, evaluation, and monitoring to better manage prompt underspecification in practice.

What Prompts Don't Say: Understanding and Managing Underspecification in LLM Prompts

TL;DR

The paper investigates prompt underspecification in large language models, showing that while models can often infer missing requirements (around 41.1% success), such behavior is unstable and sensitive to model updates and the number of explicit requirements. It demonstrates that simply specifying all requirements can hurt performance due to limited instruction-following, and that many requirements conflict or are hard to satisfy concurrently. To address this, the authors propose requirement-aware prompt optimization, including COPRO-R and a Bayesian approach, which yield about a 4.8% average accuracy improvement and, in the Bayesian case, a substantial reduction in prompt length. They advocate a systematic process for requirement elicitation, validation, and ongoing monitoring to manage prompt underspecification in practice, emphasizing drift and maintenance considerations alongside automated evaluation. Overall, the work highlights how underspecification affects reliability and offers practical optimization and governance strategies to improve robustness in real-world LLM applications.

Abstract

Prompt underspecification is a common challenge when interacting with LLMs. In this paper, we present an in-depth analysis of this problem, showing that while LLMs can often infer unspecified requirements by default (41.1%), such behavior is fragile: Under-specified prompts are 2x as likely to regress across model or prompt changes, sometimes with accuracy drops exceeding 20%. This instability makes it difficult to reliably build LLM applications. Moreover, simply specifying all requirements does not consistently help, as models have limited instruction-following ability and requirements can conflict. Standard prompt optimizers likewise provide little benefit. To address these issues, we propose requirements-aware prompt optimization mechanisms that improve performance by 4.8% on average over baselines. We further advocate for a systematic process of proactive requirements discovery, evaluation, and monitoring to better manage prompt underspecification in practice.
Paper Structure (28 sections, 16 figures, 4 tables)

This paper contains 28 sections, 16 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Underspecification is Amplified in Prompting LLMs
  3. How do LLM+Prompts Behave when Underspecified?
  4. Experiment Setups
  5. LLMs are often able to guess unspecified requirements but lack stability
  6. LLMs are more likely to regress on unspecified requirements when updated
  7. LLMs struggle with following many requirements at the same time
  8. Requirements-Aware Prompt Optimization
  9. Existing prompt optimizers do not improve performance consistently
  10. Designing requirement-aware prompt optimizers
  11. Towards Managing Prompt Underspecification
  12. Related Work
  13. Conclusion
  14. Details on Experiments Setups
  15. Task descriptions
  16. ...and 13 more sections

Figures (16)

  • Figure 1: Developers often underspecify prompts and miss user-important requirements, leading to divergent behaviors. We analyze the challenges underspecification presents and propose mechanisms to manage the problem.
  • Figure 2: While LLMs perform worse (-22.6% avg.) when a requirement is unspecified (top), they are often (41.1% avg.) able to guess unspecified requirements ($\ge0.98$ accuracy), with increased capabilities (bottom).
  • Figure 3: Cumulative distribution of accuracy drop (truncated at 0%). Prompts regress more on unspecified requirements across model updates, with an almost 2x increase compared to specified requirements.
  • Figure 4: LLMs' average accuracy on specified requirements drops with more requirements specified in the prompt, especially for smaller models like Llama-3.3-70B-Instruct.
  • Figure 5: We gather 60 requirements for our analysis. The majority of requirements come from bottom-up error analysis (41.7%), followed by existing prompts (35%), and top-down brainstorming (23.3%). Most requirements specify content-related constraints (75%), followed by style (16.7%) and format (8.3%). Most requirements are global and apply to all examples (60%), while 40% are conditional requirements. We found that existing prompts rarely consider conditional requirements (only 14.3%).
  • ...and 11 more figures